Customized tokenization was first applied to message pairs and canned replies, and then DistilBERT #REFR was used to vectorize the queries and documents for ranking.

The Smart Reply task is to take the most recent support agent message and the most recent customer message in a customer support conversation, and choose the best reply from a set of canned reply templates.

An example Smart Reply could be "This link has step-by-step instructions for how to activate Microsoft 365." An efficient CPU-based classifier is applied at runtime to classify which specific product an incoming support message interaction is for, and then our learning-to-rank model is used to select the best reply from the canned replies for that product.

Smart Reply is able to support customer support interactions across 22 Microsoft products such as Office 365, Teams, Surface, and Remote Assistance. Table 1 describes statistics for the Smart Reply task.

Training and test data were based on an 80%:20% split ratio.

One difference from traditional retrieval systems which always retrieve the top-k documents is that we do not want to overwhelm support agents with replies when there are no good canned replies for a customer message.

To achieve this, we added a "Silent" class in the product classifier and an "Empty" canned reply in each candidate reply set. We used data augmentation to generate synthetic conversations.

For instance, appending a message pair that returns the "Silent" class or "Empty" reply to non-empty questions enriched non-empty triplets, and only using agent or customer messages further enlarged the data size.

Table 2 : SR offline and online metric gains (%) Table 2 shows the 11.7% offline top-one accuracy gain of sRank compared to our previous DSSM-based #OTHEREFR system that took transformer embeddings as inputs.

We exposed Smart Reply to insider agents for initial feedback then to 50% global agents during A/B testing.

Explicit And Implicit Semantic Ranking Framework

Abstract: The core challenge in numerous real-world applications is to match an inquiry to the best document from a mutable and finite set of candidates. Existing industry solutions, especially latency-constrained services, often rely on similarity algorithms that sacrifice quality for speed. In this paper we introduce a generic semantic learning-torank framework, Self-training Semantic Cross-attention Ranking (). This transformer-based framework uses linear pairwise loss with mutable training batch sizes and achieves quality gains and high efficiency, and has been applied effectively to show gains on two industry tasks at Microsoft over real-world large-scale data sets: Smart Reply (SR) and Ambient Clinical Intelligence (ACI). In Smart Reply, assists live customers with technical support by selecting the best reply from predefined solutions based on consumer and support agent messages. It achieves 11.7% gain in offline top-one accuracy on the SR task over the previous system, and has enabled 38.7% time reduction in composing messages in telemetry recorded since its general release in January 2021. In the ACI task, selects relevant historical physician templates that serve as guidance for a text summarization model to generate higher quality medical notes. It achieves 35.5% top-one accuracy gain, along with 46% relative ROUGE-L gain in generated medical notes. CCS CONCEPTS • Information systems → Learning to rank; • Computing methodologies → Learning to rank.

Distilbert, A Distilled Version Of Bert: Smaller, Faster, Cheaper And Lighter

Abstract: As Transfer Learning from large-scale pre-trained models becomes more prevalent in Natural Language Processing (NLP), operating these large models in on-the-edge and/or under constrained computational training or inference budgets remains challenging. In this work, we propose a method to pre-train a smaller general-purpose language representation model, called DistilBERT, which can then be fine-tuned with good performances on a wide range of tasks like its larger counterparts. While most prior work investigated the use of distillation for building task-specific models, we leverage knowledge distillation during the pre-training phase and show that it is possible to reduce the size of a BERT model by 40%, while retaining 97% of its language understanding capabilities and being 60% faster. To leverage the inductive biases learned by larger models during pre-training, we introduce a triple loss combining language modeling, distillation and cosine-distance losses. Our smaller, faster and lighter model is cheaper to pre-train and we demonstrate its capabilities for on-device computations in a proof-of-concept experiment and a comparative on-device study.


With R redefined to have a sparse complementa string is in R unless exponentially compressible-the complement of the #REFR .

#OTHEREFR We show: if it is possible to efficiently rule out length t proofs of some unprovable sentence φ, it is also possible to efficiently rule out a slightly shorter proof of inconsistency, which could be used in a length t proof of φ by contradiction.

This implies a powerful generalization-if it is hard to rule out length t proofs of inconsistency, it is hard to rule of length t proofs of any unprovable sentence.

This in turn implies that facts about unprovability and noncomputability, which are well understood, can be imported into complexity theory.

This has wide ramifications-diverse types of unprovable sentences translate into assertions that open questions in complexity theory have the expected answers.

For instance, unprovable sentences of the form x∈R are dense, so hard families of tautologies encoding "no length t proof shows x∈R" are also dense.

Pudlák #OTHEREFR shows the initial conjecture was incorrect-a theory T can efficiently prove that T lacks a length t proof of '0=1'.

The 1989 reformulation refers to the lack of efficient proofs in a weaker theory. See also Theorem 59 of Pudlák #OTHEREFR .

3 See also Krajíček #OTHEREFR Section 21.3.

language { x, 1 t | no length t proof exists of x∈R} is neither in P nor NPcomplete, but is NP-intermediate.

The hardness of ruling out length t proofs of any unprovable sentence implies a deep linkage between noncomputability and complexity.

Ruling Out Short Proofs Of Unprovable Sentences Is Hard

Abstract: If no optimal propositional proof system exists, we (and independently Pudlák) prove that ruling out length t proofs of any unprovable sentence is hard. This mapping from unprovable to hard-to-prove sentences powerfully translates facts about noncomputability into complexity theory. For instance, because proving string x is Kolmogorov random (x∈R) is typically impossible, it is typically hard to prove "no length t proof shows x∈R", or tautologies encoding this. Therefore, a proof system with one family of hard tautologies has these densely in an enumeration of families. The assumption also implies that a natural language is NP-intermediate: with R redefined to have a sparse complement, the complement of the language { x, 1 t | no length t proof exists of x∈R} is also sparse. Efficiently ruling out length t proofs of x∈R might violate the constraint on using the fact of x∈R's unprovability. We conjecture: any computable predicate on R that might be used in if-then statements (or case-based proofs) does no better than branching at random, because R appears random by any effective test. This constraint could also inhibit the usefulness in circuits and propositional proofs of NOT gates and cancellation-needed to encode if-then statements. If R defeats if-then logic, exhaustive search is necessary.

Incompleteness In The Finite Domain

Abstract: Motivated by the problem of finding finite versions of classical incompleteness theorems, we present some conjectures that go beyond ${\bf NP\neq co NP}$. These conjectures formally connect computational complexity with the difficulty of proving some sentences, which means that high computational complexity of a problem associated with a sentence implies that the sentence is not provable in a weak theory, or requires a long proof. Another reason for putting forward these conjectures is that some results in proof complexity seem to be special cases of such general statements and we want to formalize and fully understand these statements. In this paper we review some conjectures that we have presented earlier, introduce new conjectures, systematize them and prove new connections between them and some other statements studied before.


Theorem 2.8 ([Hlw94, Hau95]). Fix A Hypothesis Class

Corollary 2.9 recovers the known optimal risk upper bound for binary classification first proven by Hanneke #REFR .

Fix a hypothesis class H ⊆ {0, 1} X with VC dimension d.

There is a predictor f : X × U → {0, 1} which, for any δ ∈ (0, 1), and S ∼ P n sampled from any realizable distribution P , satisfies

err P ( f (•; S)) 9.64 d n + 1 n log 2 δ ,

with probability at least 1 − δ over the randomness of S.

Recently, Larsen #OTHEREFR showed that an implementation of the natural bagging heuristic also achieves an optimal risk bound.

Our proof of the optimal bound is remarkably simpler than both of their proofs.

Furthermore, both Hanneke and Larsen state that the constant factors in their upper bounds are very large and explicitly ask whether these constants can be reduced.

Our new analysis reduces the constant factors by a few orders of magnitude.

In addition, our result can be seen as a partial answer to a question of Warmuth #OTHEREFR who asked whether the one-inclusion graph algorithm can achieve an optimal PAC risk bound.

Optimal Pac Bounds Without Uniform Convergence

Abstract: In statistical learning theory, determining the sample complexity of realizable binary classification for VC classes was a long-standing open problem. The results of Simon [Sim15] and Hanneke [Han16a] established sharp upper bounds in this setting. However, the reliance of their argument on the uniform convergence principle limits its applicability to more general learning settings such as multiclass classification. In this paper, we address this issue by providing optimal high probability risk bounds through a framework that surpasses the limitations of uniform convergence arguments. Our framework converts the leave-one-out error of permutation invariant predictors into high probability risk bounds. As an application, by adapting the one-inclusion graph algorithm of Haussler, Littlestone, and Warmuth [HLW94], we propose an algorithm that achieves an optimal PAC bound for binary classification. Specifically, our result shows that certain aggregations of one-inclusion graph algorithms are optimal, addressing a variant of a classic question posed by Warmuth [War04]. We further instantiate our framework in three settings where uniform convergence is provably suboptimal. For multiclass classification, we prove an optimal risk bound that scales with the one-inclusion hypergraph density of the class, addressing the suboptimality of the analysis of Daniely and Shalev-Shwartz [DS14]. For partial hypothesis classification, we determine the optimal sample complexity bound, resolving a question posed by Alon, Hanneke, Holzman, and Moran [AHHM22]. For realizable bounded regression with absolute loss, we derive an optimal risk bound that relies on a modified version of the scale-sensitive dimension, refining the results of Bartlett and Long [BL98]. Our rates surpass standard uniform convergence-based results due to the smaller complexity measure in our risk bound.

The Optimal Sample Complexity Of Pac Learning

Abstract: This work establishes a new upper bound on the number of samples sufficient for PAC learning in the realizable case. The bound matches known lower bounds up to numerical constant factors. This solves a long-standing open problem on the sample complexity of PAC learning. The technique and analysis build on a recent breakthrough by Hans Simon.


Incremental Classifier and Presentation Learning (iCaRL) #REFR performs classification using the nearest mean-of-exemplars, where the exemplars selected by herding algorithm in the feature space.

In our evaluation, we test our method by combing it with two popular experience replay methods, ER #OTHEREFR and DER++ #OTHEREFR .

ER uses a memory buffer to store the training examples from past tasks and interleaves them with the current task data for training.

In addition to this, DER++ records the output logits of the examples in the memory and performs logit distillation when doing experience replay.

We combine the proposed BFP loss with ER and DER++ and denote them as ER w/ BFP and DER++ w/ BFP respectively.

We also compare the proposed method with some other state-of-the-art CL baselines as listed in Table 1.

Functional Distance Regularization (FDR) #OTHEREFR regularize the output of the network to its past value.

Different from DER/DER++, FDR applies the regularization on the output classification probability.

Learning a Unified Classifier Incrementally via Rebalancing (LUCIR) #OTHEREFR augments experience replay with multiple modifications to preserve old knowledge and enforce separation class separation in continual learning.

Bias Correction (BiC) #OTHEREFR augments the experience replay by learning a separate layer to correct the bias in the output logits.

ER with Asymmetric Cross-Etronpy (ER-ACE) #OTHEREFR proposes to reduce representation drift by using separate cross-entropy loss for online and replayed training data.

Preserving Linear Separability In Continual Learning By Backward Feature Projection

Abstract: Catastrophic forgetting has been a major challenge in continual learning, where the model needs to learn new tasks with limited or no access to data from previously seen tasks. To tackle this challenge, methods based on knowledge distillation in feature space have been proposed and shown to reduce forgetting [16, 18, 26]. However, most feature distillation methods directly constrain the new features to match the old ones, overlooking the need for plasticity. To achieve a better stability-plasticity trade-off, we propose Backward Feature Projection (BFP), a method for continual learning that allows the new features to change up to a learnable linear transformation of the old features. BFP preserves the linear separability of the old classes while allowing the emergence of new feature directions to accommodate new classes. BFP can be integrated with existing experience replay methods and boost performance by a significant margin. We also demonstrate that BFP helps learn a better representation space, in which linear separability is well preserved during continual learning and linear probing achieves high classification accuracy. The code can be found at https://github.com/rvl-lab-utoronto/BFP.

Icarl: Incremental Classifier And Representation Learning

Abstract: A major open problem on the road to artificial intelligence is the development of incrementally learning systems that learn about more and more concepts over time from a stream of data. In this work, we introduce a new training strategy, iCaRL, that allows learning in such a class-incremental way: only the training data for a small number of classes has to be present at the same time and new classes can be added progressively. iCaRL learns strong classifiers and a data representation simultaneously. This distinguishes it from earlier works that were fundamentally limited to fixed data representations and therefore incompatible with deep learning architectures. We show by experiments on CIFAR-100 and ImageNet ILSVRC 2012 data that iCaRL can learn many classes incrementally over a long period of time where other strategies quickly fail.


Additionally, our results demonstrate meaningful gains over Turbo Autoencoder (TurboAE) #REFR and state-of-the-art classical codes.

In other words, ProductAE boils down the complex problem of training autoencoders of large dimensions and lengths to less-complex sub-problems of training encoders and decoders for smaller dimensions and lengths.

Our training results for a relatively short-length ProductAE of dimension k = 100 show significant performance gains compared to the polar code under successive cancellation (SC) decoding.

More importantly, we demonstrate achieving similar gains for moderate-length ProductAEs of dimensions as large as k = 300 bits.

This clearly establishes the generalization of our proposed architecture for training higher-dimension codes.

These achievements are attained by applying innovative ideas from deep learning and intuitions from coding theory to further improve the training performance of our proposed ProductAE architecture.

The main contributions of the paper are summarized as follows.

• We introduce a new class of neural error-correction codes, namely ProductAE, aimed at enabling the training of higher-dimension codes.

Building upon ideas from classical product codes, ProductAE boils down the complex problem of training large autoencoders to less-complex sub-problems of training smaller encoding and decoding components.

• We present several useful modifications to our proposed ProductAE architecture, which significantly contribute to achieving excellent training performances.

Productae: Toward Deep Learning Driven Error-Correction Codes Of Large Dimensions

Abstract: While decades of theoretical research have led to the invention of several classes of error-correction codes, the design of such codes is an extremely challenging task, mostly driven by human ingenuity. Recent studies demonstrate that such designs can be effectively automated and accelerated via tools from machine learning (ML), thus enabling ML-driven classes of errorcorrection codes with promising performance gains compared to classical designs. A fundamental challenge, however, is that it is prohibitively complex, if not impossible, to design and train fully ML-driven encoder and decoder pairs for large code dimensions. In this paper, we propose Product Autoencoder (ProductAE)-a computationally-efficient family of deep learning driven (encoder, decoder) pairs-aimed at enabling the training of relatively large codes (both encoder and decoder) with a manageable training complexity. We build upon ideas from classical product codes and propose constructing large neural codes using smaller code components. ProductAE boils down the complex problem of training the encoder and decoder for a large code dimension k and blocklength n to less-complex sub-problems of training encoders and decoders for smaller dimensions and blocklengths. Our training results show successful training of ProductAEs of dimensions as large as k = 300 bits with meaningful performance gains compared to state-of-the-art classical and neural designs. Moreover, we demonstrate excellent robustness and adaptivity of ProductAEs to channel models different than the ones used for training.

Turbo Autoencoder: Deep Learning Based Channel Codes For Point-To-Point Communication Channels

Abstract: Designing codes that combat the noise in a communication medium has remained a significant area of research in information theory as well as wireless communications. Asymptotically optimal channel codes have been developed by mathematicians for communicating under canonical models after over 60 years of research. On the other hand, in many non-canonical channel settings, optimal codes do not exist and the codes designed for canonical models are adapted via heuristics to these channels and are thus not guaranteed to be optimal. In this work, we make significant progress on this problem by designing a fully end-to-end jointly trained neural encoder and decoder, namely, Turbo Autoencoder (TurboAE), with the following contributions: ($a$) under moderate block lengths, TurboAE approaches state-of-the-art performance under canonical channels; ($b$) moreover, TurboAE outperforms the state-of-the-art codes under non-canonical settings in terms of reliability. TurboAE shows that the development of channel coding design can be automated via deep learning, with near-optimal performance.


Effects Of Different Window Size W In Dsbe (Section Iii-C).

Our experimental results on Pancreas dataset show that changing one base model to 3D U-Net #REFR results in similar results to those reported in Table III : 73.89±4.42 vs. 74.03±4.00.

That is, the pseudo-labels generated by its peer networks will be exactly the same as the predictions made by the base model, and each base model independently learns from its own predictions in every iteration, which is equivalent to Entropy Minimization.

The only difference between the degraded ComWin model and regular Entropy Minimization, is that the former fits a categorical probability distribution while the latter learns from a continuous soft counterpart. This disparity is minor.

As shown in Table III , switching to Entropy Minimization would incur a huge performance drop (from 74.03 to 44.55) .

This demonstrates the significance of initializing each base model differently. Base model architectures.

This framework is structure agnostic and does not limit the choices of architecture configurations.

Thorough exploration regarding diversity and the combination of base model structures would be an interesting direction for future studies.

Compete To Win: Enhancing Pseudo Labels For Barely-Supervised Medical Image Segmentation

Abstract: This study investigates barely-supervised medical image segmentation where only few labeled data, i.e., singledigit cases are available. We observe the key limitation of the existing state-of-the-art semi-supervised solution cross pseudo supervision is the unsatisfactory precision of foreground classes, leading to a degenerated result under barely-supervised learning. In this paper, we propose a novel Compete-to-Win method (ComWin) to enhance the pseudo label quality. In contrast to directly using one model's predictions as pseudo labels, our key idea is that high-quality pseudo labels should be generated by comparing multiple confidence maps produced by different networks to select the most confident one (a compete-to-win strategy). To further refine pseudo labels at near-boundary areas, an enhanced version of ComWin, namely, ComWin + , is proposed by integrating a boundary-aware enhancement module. Experiments show that our method can achieve the best performance on three public medical image datasets for cardiac structure segmentation, pancreas segmentation and colon tumor segmentation, respectively. The source code is now available at https://github.com/Huiimin5/comwin.

U-Net: Convolutional Networks For Biomedical Image Segmentation

Abstract: There is large consent that successful training of deep networks requires many thousand annotated training samples. In this paper, we present a network and training strategy that relies on the strong use of data augmentation to use the available annotated samples more efficiently. The architecture consists of a contracting path to capture context and a symmetric expanding path that enables precise localization. We show that such a network can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks. Using the same network trained on transmitted light microscopy images (phase contrast and DIC) we won the ISBI cell tracking challenge 2015 in these categories by a large margin. Moreover, the network is fast. Segmentation of a 512x512 image takes less than a second on a recent GPU. The full implementation (based on Caffe) and the trained networks are available at http://lmb.informatik.uni-freiburg.de/people/ronneber/u-net .


This is in line with other results in the literature #REFR and is related to the trade-off between model depth and GPU compute units available.

In all cases, we use the authors' provided code, but with a modification to accept randomly generated video sequences of two different durations.

The results can be seen in Table 4 and show that our MobileVOS models are significantly smaller, while retaining a constant low latency across both the short and long-duration videos.

This result is attributed to the smaller backbones and the constant memory costs due to a finite memory queue length.

Our largest model with ResNet18 has more than 3ˆthe FPS than RDE-VOS but with 8f ewer parameters.

Finally, we observe that the ResNet models perform better on the server-grade GPUs.

In the following section, we show that we obtain a different outcome for mobile GPUs. Table 4 . Performance evaluation on the same sets of hardware.

The FPS metrics were evaluated on two randomly generated short and long video sequences with shape 480ˆ910.

The short videos consist of 50 frames, while the long videos consist of 500.

In all cases, we use the authors' provided code and indicates that the models are exceeding the GPU memory limit.

Mobilevos: Real-Time Video Object Segmentation Contrastive Learning Meets Knowledge Distillation

Abstract: This paper tackles the problem of semi-supervised video object segmentation on resource-constrained devices, such as mobile phones. We formulate this problem as a distillation task, whereby we demonstrate that small spacetime-memory networks with finite memory can achieve competitive results with state of the art, but at a fraction of the computational cost (32 milliseconds per frame on a Samsung Galaxy S22). Specifically, we provide a theoretically grounded framework that unifies knowledge distillation with supervised contrastive representation learning. These models are able to jointly benefit from both pixel-wise contrastive learning and distillation from a pretrained teacher. We validate this loss by achieving competitive J &F to state of the art on both the standard DAVIS and YouTube benchmarks, despite running up toˆ5 faster, and withˆ32 fewer parameters.

Benchmark Analysis Of Representative Deep Neural Network Architectures

Abstract: This work presents an in-depth analysis of the majority of the deep neural networks (DNNs) proposed in the state of the art for image recognition. For each DNN multiple performance indices are observed, such as recognition accuracy, model complexity, computational complexity, memory usage, and inference time. The behavior of such performance indices and some combinations of them are analyzed and discussed. To measure the indices we experiment the use of DNNs on two different computer architectures, a workstation equipped with a NVIDIA Titan X Pascal and an embedded system based on a NVIDIA Jetson TX1 board. This experimentation allows a direct comparison between DNNs running on machines with very different computational capacity. This study is useful for researchers to have a complete view of what solutions have been explored so far and in which research directions are worth exploring in the future; and for practitioners to select the DNN architecture(s) that better fit the resource constraints of practical deployments and applications. To complete this work, all the DNNs, as well as the software used for the analysis, are available online.


Since our annotations have been checked manually, we do not perform a bootstrapping procedure as #REFR .

Our dataset is of single hands and we apply 21 keypoints scheme #OTHEREFR to annotate each hand.

Inspired by Interhand2.6M #OTHEREFR and FreiHand #OTHEREFR , we use multi-view RGB images for 3D annotation and apply a two-stage process consisting of machine annotation and human annotation.

First, we use Mediapipe [47] to detect 2D hand keypoints on all the RGB images and triangulate 2D keypoints to obtain 3D keypoints with RANSAC method.

Then we manually verify all the keypoints re-projected by 3D keypoints and select the unqualified views for human annotation.

Due to the cost of human annotation, manual annotation is only applied to fixed pose and random short sequences and machine annotation is then applied to random long sequences.

For fast motion sequences, we can not get accurate 3D joint annotations due to the severe motion blur in the captured images.

To quantitatively evaluate our method, we manually annotate the 2D joints on the event sequences.

Evhandpose: Event-Based 3D Hand Pose Estimation With Sparse Supervision

Abstract: Event camera shows great potential in 3D hand pose estimation, especially addressing the challenges of fast motion and high dynamic range in a low-power way. However, due to the asynchronous differential imaging mechanism, it is challenging to design event representation to encode hand motion information especially when the hands are not moving (causing motion ambiguity), and it is infeasible to fully annotate the temporally dense event stream. In this paper, we propose EvHandPose with novel hand flow representations in Event-to-Pose module for accurate hand pose estimation and alleviating the motion ambiguity issue. To solve the problem under sparse annotation, we design contrast maximization and edge constraints in Pose-to-IWE (Image with Warped Events) module and formulate EvHandPose in a self-supervision framework. We further build EvRealHands, the first large-scale real-world event-based hand pose dataset on several challenging scenes to bridge the domain gap due to relying on synthetic data and facilitate future research. Experiments on EvRealHands demonstrate that EvHandPose outperforms previous event-based method under all evaluation scenes with 15 ∼ 20 mm lower MPJPE and achieves accurate and stable hand pose estimation in fast motion and strong light scenes compared with RGB-based methods. Furthermore, EvHandPose demonstrates 3D hand pose estimation at 120 fps or higher.

Hand Keypoint Detection In Single Images Using Multiview Bootstrapping

Abstract: We present an approach that uses a multi-camera system to train fine-grained detectors for keypoints that are prone to occlusion, such as the joints of a hand. We call this procedure multiview bootstrapping: first, an initial keypoint detector is used to produce noisy labels in multiple views of the hand. The noisy detections are then triangulated in 3D using multiview geometry or marked as outliers. Finally, the reprojected triangulations are used as new labeled training data to improve the detector. We repeat this process, generating more labeled data in each iteration. We derive a result analytically relating the minimum number of views to achieve target true and false positive rates for a given detector. The method is used to train a hand keypoint detector for single images. The resulting keypoint detector runs in realtime on RGB images and has accuracy comparable to methods that use depth sensors. The single view detector, triangulated over multiple views, enables 3D markerless hand motion capture with complex object interactions.


The shortcomings of standard RC algorithm on C-V2X has been explored in #REFR , which is highly suggested for readers interested in congestion control approaches in C-V2X.

Mean ITT for the tested scenarios is presented in figure 4 , which heavily correlates with the CBR at corresponding densities.

At very low traffic (1veh/s), 35% CBR (figure 3) causes RC to remain dormant, resulting in VUEs transmitting BSMs with 100ms ITT (figure 4).

But a slight increase of traffic flow (5veh/s) vastly increases the CBR to 90%.

This change is not reflected comparably at ITT, which barely increases to 110ms.

This low sensitivity of RC algorithm at low traffic flow appears from the algorithm originally being optimized for the older DSRC technology, which follows a different abstraction of physical layer bandwidth than C-V2X.

Further increase in density (10 veh/s) causes gradual increase in ITT.

With higher ITT, larger volume of resources are available within a unit time-frame due to less frequent transmission by the VUEs.

As a result, CBR comparably decreases and the now-accessible resources are utilized by the service application, making SCT at 10veh/s better than 5veh/s (figure 2).

At high CBR, SPS resource allocation procedure is more aggressive in resource shortlisting and selection.

When a VUE transmitting in a high density traffic scenario increases its SPS RSRP threshold while shortlisting a resource for its own transmission, one implication of this occurrence is that the VUE can likely select a resource with low RSRP for its upcoming transmission.

On Batching Acknowledgements In C-V2X Services

Abstract: Cellular Vehicle-to-Everything (C-V2X) is a frontier in the evolution of distributed communication introduced in 3GPP release 14 to advanced use cases. While research efforts continue to optimize the accessible bandwidth for transportation ecosystem, a bottom up analysis from the application layer perspective is necessary prior to deployment, as it can expose potential issues that can emerge in a dynamic road environment. This emphasizes on assessing the network using applicationoriented metrics to evaluate its capacity of providing advanced vehicular services with stringent latency and throughput requirements. C-V2X enables advanced applications like autonomous driving and on-the-go transaction services where consecutive exchange of messages is required. For such services, the network level metrics fails to capture the edge case service quality as they express an average measure of performance. In this paper, we present an application-oriented analysis of a transaction service built on C-V2X protocol. We analyze different design choices that affects quality of service both from network-oriented and user-centric metrics and we highlight the issues regarding packet dissemination from infrastructures for vehicle-to-infrastructure (V2I) based service applications. We also present our study on the impact of batching in disseminating acknowledgement packets (ACK) and its consequence on both the service reliability and network congestion. Our results show that time-sensitive and mission-sensitive vehicular applications should aim for a balance between achieving the mission utility in shortest duration possible, while keeping minimal impact on the system-wide stability.

Analysis Of Distributed Congestion Control In Cellular Vehicle-To-Everything Networks

Abstract: Cellular Vehicle-to-everything (C-V2X) communication has been proposed in the 3rd Generation Partnership Project release 14 standard to address the latency and reliability requirements of cooperative safety applications. Such applications can involve highly congested vehicular scenarios where the network experiences high data loads. Thus, a sophisticated congestion control solution is vital in order to maintain the network performance required for safety-related applications. With the aid of our high-fidelity link-level network simulator, we investigate the feasibility of implementing the distributed congestion control algorithm specified in SAE J2945/1 standard on top of the C-V2X stack. We describe our implementation and evaluate the performance of transmission rate and range control mechanisms using relevant metrics. Additionally, we identify areas for potential design enhancements and further investigation.


Specifically, the Cross-Modal Traction module is composed of a stack of N blocks, and in each block, we first consider a scaled dot product attention mechanism #REFR with the prompt representation attending to all patch tokens.

Given the same semantic concept, the static and monotonous prompt representation naturally fails to be commonly optimal for all input images that come from a plentiful distribution.

This issue becomes more serious in the additional state and object branches, as the visual content of the same primitive changes considerably when paired with different primitives.

Therefore, we further develop a Cross-Modal Traction module for Troika.

The module adaptively shifts the prompt representation to accommodate the content diversity and diminish the cross-modal discrepancies.

In this process, relevant patch features serve as the guidance to avoid noise from semantic-agnostic sub-regions interfering with the traction.

Given the input prompt representation t that comes from an arbitrary branch, we first acquire the patch tokens X p ∈ R N p ×d after projecting them with the linear layer g proj .

Then, the query, key and value can be derived as

where W q , W K , W V ∈ R d×d attn are the parameter matrices, and d attn is the dimension of the single-head attention.

The dot product attention gives relevance weights from t to each patch token, which are used to aggregate the value-projected patch tokens as

Troika: Multi-Path Cross-Modal Traction For Compositional Zero-Shot Learning

Abstract: Recent compositional zero-shot learning (CZSL) methods adapt pre-trained vision-language models (VLMs) by constructing trainable prompts only for composed state-object pairs. Relying on learning the joint representation of seen compositions, these methods ignore the explicit modeling of the state and object, thus limiting the exploitation of pre-trained knowledge and generalization to unseen compositions. With a particular focus on the universality of the solution, in this work, we propose a novel paradigm for CZSL models that establishes three identification branches (i.e., Multi-Path) to jointly model the state, object, and composition. The presented Troika is our implementation that aligns the branch-specific prompt representations with decomposed visual features. To calibrate the bias between semantically similar multi-modal representations, we further devise a Cross-Modal Traction module into Troika that shifts the prompt representation towards the current visual content. We conduct extensive experiments on three popular benchmarks, where our method significantly outperforms existing methods in both closed-world and open-world settings.

Attention Is All You Need

Abstract: The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.

prompt representation
attention mechanism


To date, most of the topperforming IS methods still follow the Mask R-CNN metaarchitecture #REFR .

These proposal-based approaches typically employ an object detector to localize each instance in bounding boxes.

Then the instance-wise features are cropped and extracted from FPN features based on the detected bounding boxes by using RoI pooling/align #OTHEREFR .

Finally, a compact segmentation head is deployed to obtain the desired object masks.

Mask Scoring R-CNN #OTHEREFR aligns the mask quality and score by using a branch to explicitly learn the quality of predicted masks.

BMask R-CNN #OTHEREFR leverages boundary details to improve mask localization ability.

Dynamask: Dynamic Mask Selection For Instance Segmentation

Abstract: The representative instance segmentation methods mostly segment different object instances with a mask of the fixed resolution, e.g., 28 × 28 grid. However, a lowresolution mask loses rich details, while a high-resolution mask incurs quadratic computation overhead. It is a challenging task to predict the optimal binary mask for each instance. In this paper, we propose to dynamically select suitable masks for different object proposals. First, a dual-level Feature Pyramid Network (FPN) with adaptive feature aggregation is developed to gradually increase the mask grid resolution, ensuring high-quality segmentation of objects. Specifically, an efficient region-level top-down path (r-FPN) is introduced to incorporate complementary contextual and detailed information from different stages of image-level FPN (i-FPN). Then, to alleviate the increase of computation and memory costs caused by using large masks, we develop a Mask Switch Module (MSM) with negligible computational cost to select the most suitable mask resolution for each instance, achieving high efficiency while maintaining high segmentation accuracy. Without bells and whistles, the proposed method, namely DynaMask, brings consistent and noticeable performance improvements over other state-ofthe-arts at a moderate computation overhead. The source code: https://github.com/lslrh/DynaMask.

Mask R-Cnn

Abstract: We present a conceptually simple, flexible, and general framework for object instance segmentation. Our approach efficiently detects objects in an image while simultaneously generating a high-quality segmentation mask for each instance. The method, called Mask R-CNN, extends Faster R-CNN by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recognition. Mask R-CNN is simple to train and adds only a small overhead to Faster R-CNN, running at 5 fps. Moreover, Mask R-CNN is easy to generalize to other tasks, e.g., allowing us to estimate human poses in the same framework. We show top results in all three tracks of the COCO suite of challenges, including instance segmentation, bounding-box object detection, and person keypoint detection. Without bells and whistles, Mask R-CNN outperforms all existing, single-model entries on every task, including the COCO 2016 challenge winners. We hope our simple and effective approach will serve as a solid baseline and help ease future research in instance-level recognition. Code has been made available at: https://github.com/facebookresearch/Detectron .


The Contextual Loss (CX) #REFR is a viable choice as it treats features of images as a set and measures the similarity between images, ignoring the spatial positions of the features.

The training of AlignFormer requires an objective function that does not forcefully match each spatial position as the problem lacks exact spatially aligned supervision.

This property enables us to compare images that are spatially deformed.

Given two images x and y, CX loss aims to minimize the summed distance of all matched feature pairs, formulated as

where ϕ(x) j and ϕ(y) i are the j-th point of ϕ(x) and ith point of ϕ(y), respectively.

ϕ(x) denotes feature maps of x extracted from the VGG network ϕ, and D is some distance measure.

Generating Aligned Pseudo-Supervision From Non-Aligned Data For Image Restoration In Under-Display Camera

Abstract: Due to the difficulty in collecting large-scale and perfectly aligned paired training data for Under-Display Camera (UDC) image restoration, previous methods resort to monitor-based image systems or simulation-based methods, sacrificing the realness of the data and introducing domain gaps. In this work, we revisit the classic stereo setup for training data collection-capturing two images of the same scene with one UDC and one standard camera. The key idea is to "copy" details from a high-quality reference image and "paste" them on the UDC image. While being able to generate real training pairs, this setting is susceptible to spatial misalignment due to perspective and depth of field changes. The problem is further compounded by the large domain discrepancy between the UDC and normal images, which is unique to UDC restoration. In this paper, we mitigate the non-trivial domain discrepancy and spatial misalignment through a novel Transformer-based framework that generates well-aligned yet high-quality target data for the corresponding UDC input. This is made possible through two carefully designed components, namely, the Domain Alignment Module (DAM) and Geometric Alignment Module (GAM), which encourage robust and accurate discovery of correspondence between the UDC and normal views. Extensive experiments show that high-quality and well-aligned pseudo UDC training pairs are beneficial for training a robust restoration network. Code and the dataset are available at https://github.com/ jnjaby/AlignFormer.

The Contextual Loss For Image Transformation With Non-Aligned Data

Abstract: Feed-forward CNNs trained for image transformation problems rely on loss functions that measure the similarity between the generated image and a target image. Most of the common loss functions assume that these images are spatially aligned and compare pixels at corresponding locations. However, for many tasks, aligned training pairs of images will not be available. We present an alternative loss function that does not require alignment, thus providing an effective and simple solution for a new space of problems. Our loss is based on both context and semantics -- it compares regions with similar semantic meaning, while considering the context of the entire image. Hence, for example, when transferring the style of one face to another, it will translate eyes-to-eyes and mouth-to-mouth. Our code can be found at https://www.github.com/roimehrez/contextualLoss


While this is not the focus of this work, our finding suggests that synthetic samples can be helpful for accuracy if used properly, challenging previous beliefs #REFR .

On CIFAR-10 far-OOD detection, SIO does not outperform vanilla OE, which incorporates a large set of external OOD samples for training.

We suspect that this is because far-OOD detection on CIFAR-10 is dominated by low-level statistics #OTHEREFR , and using synthetic ID samples may slightly push the model away from the real low-level statistics of the ID data, causing a shrinked difference between ID and OOD samples.

Nonetheless, SIO improves far-OOD detection in all other cases and effectively closes the gap between non-OE methods and OE.

Notably, LogitNorm + SIO yields a 97.09% AUROC without any OOD training data, which is on par with the 97.16% AUROC achieved by OE.

Lastly, we observe that SIO can also benefit ID classification accuracy.

On the other hand, however, in our later experiments where we vary the hyperparameters, we find that SIO consistently boosts OOD detection performance even if it does not improve ID accuracy.

More discussion on this can be found in Section 4.3.

On CIFAR-100, we observe similar results to CIFAR-10, with the general observation that SIO can benefit OOD detection performance.

With SIO, the average near-OOD / far-OOD AUROC is lifted from 76.63% / 80.29% to 77.20% / 83.65%, and the best numbers are improved from 80.22% / 87.54% to 80.34% / 89.37%.

Sio: Synthetic In-Distribution Data Benefits Out-Of-Distribution Detection

Abstract: Building up reliable Out-of-Distribution (OOD) detectors is challenging, often requiring the use of OOD data during training. In this work, we develop a data-driven approach which is distinct and complementary to existing works: Instead of using external OOD data, we fully exploit the internal in-distribution (ID) training set by utilizing generative models to produce additional synthetic ID images. The classifier is then trained using a novel objective that computes weighted loss on real and synthetic ID samples together. Our training framework, which is termed SIO, serves as a "plug-and-play" technique that is designed to be compatible with existing and future OOD detection algorithms, including the ones that leverage available OOD training data. Our experiments on CIFAR-10, CIFAR-100, and ImageNet variants demonstrate that SIO consistently improves the performance of nearly all stateof-the-art (SOTA) OOD detection algorithms. For instance, on the challenging CIFAR-10 v.s. CIFAR-100 detection problem, SIO improves the average OOD detection AUROC of 18 existing methods from 86.25% to 89.04% and achieves a new SOTA of 92.94% according to the OpenOOD benchmark. Code is available at https://github.com/ zjysteven/SIO.

Classification Accuracy Score For Conditional Generative Models

Abstract: Deep generative models (DGMs) of images are now sufficiently mature that they produce nearly photorealistic samples and obtain scores similar to the data distribution on heuristics such as Frechet Inception Distance (FID). These results, especially on large-scale datasets such as ImageNet, suggest that DGMs are learning the data distribution in a perceptually meaningful space and can be used in downstream tasks. To test this latter hypothesis, we use class-conditional generative models from a number of model classes—variational autoencoders, autoregressive models, and generative adversarial networks (GANs)—to infer the class labels of real data. We perform this inference by training an image classifier using only synthetic data and using the classifier to predict labels on real data. The performance on this task, which we call Classification Accuracy Score (CAS), reveals some surprising results not identified by traditional metrics and constitute our contributions. First, when using a state-of-the-art GAN (BigGAN-deep), Top-1 and Top-5 accuracy decrease by 27.9% and 41.6%, respectively, compared to the original data; and conditional generative models from other model classes, such as Vector-Quantized Variational Autoencoder-2 (VQ-VAE-2) and Hierarchical Autoregressive Models (HAMs), substantially outperform GANs on this benchmark. Second, CAS automatically surfaces particular classes for which generative models failed to capture the data distribution, and were previously unknown in the literature. Third, we find traditional GAN metrics such as Inception Score (IS) and FID neither predictive of CAS nor useful when evaluating non-GAN models. Furthermore, in order to facilitate better diagnoses of generative models, we open-source the proposed metric.


Also, article #REFR point that the methods developed in the ML have been particularly successful in big data settings, where we observe information on a large number of units, many pieces of information on each unit, or both.

Lastly, large datasets allow for more flexible relationships than simple linear models can capture.

ML techniques are handy in those cases due to their ability to model intricate and nonlinear relationships potentially offering new insights.

Similarly, article #OTHEREFR argue that ML not only provides new tools but also solves a different problem.

They assert ML's success is largely due to its ability to discover the complex structure that was not specified in advance.

They suggest applying ML to economics requires finding relevant tasks, for instance, where the focus is on increasing prediction accuracy or uncovering generalizable patterns from complex datasets.

The authors suggest that for using ML tools for economics research and analysis, researchers should clearly articulate their goals and why certain properties of ML algorithms may or may not be important.

Machine Learning For Economics Research: When What And How? *

Abstract: This article provides a curated review of selected papers published in prominent economics journals that use machine learning (ML) tools for research and policy analysis. The review focuses on three key questions: (1) when ML is used in economics, (2) what ML models are commonly preferred, and (3) how they are used for economic applications. The review highlights that ML is particularly used to process nontraditional and unstructured data, capture strong nonlinearity, and improve prediction accuracy. Deep learning models are suitable for nontraditional data, whereas ensemble learning models are preferred for traditional datasets. While traditional econometric models may suffice for analyzing low-complexity data, the increasing complexity of economic data due to rapid digitalization and the growing literature suggests that ML is becoming an essential addition to the econometrician's toolbox.

Machine Learning Methods Economists Should Know About

Abstract: We discuss the relevance of the recent Machine Learning (ML) literature for economics and econometrics. First we discuss the differences in goals, methods and settings between the ML literature and the traditional econometrics and statistics literatures. Then we discuss some specific methods from the machine learning literature that we view as important for empirical researchers in economics. These include supervised learning methods for regression and classification, unsupervised learning methods, as well as matrix completion methods. Finally, we highlight newly developed methods at the intersection of ML and econometrics, methods that typically perform better than either off-the-shelf ML or more traditional econometric methods when applied to particular classes of problems, problems that include causal inference for average treatment effects, optimal policy estimation, and estimation of the counterfactual effect of price changes in consumer choice models.


If any iteration fails to converge in 512 integration steps, then the spin output s is such that Ts = 0 (where T is the transform matrix defined by #REFR ).

For the scenarios simulated in this paper, dt = 0.005 was found to be sufficient.

The initial values x i are i.i.d N (0, 0.001) and e i are initialized using a folded N (0, 0.001) distribution.

These parameters were empirically selected, based on trial-and-error experiments, such that the system can achieve a steady state and attain good performance.

Note that performance can be further improved by optimally selecting these parameters, and we plan to address this in our future work.

2) Evaluation Metrics: Our evaluation setup simulates an uplink N t × N r MIMO system which has N t users (with one transmit antenna each) and N r receive antennas at the base station (N r > N t ).

We assume a slow-fading channel, and channel instances are assumed to follow the Rayleigh fading model.

The BER is computed as the mean BER of all users.

We compare our methods against MMSE-SIC with optimal ordering #OTHEREFR and the MMSE detector in both large and massive MIMO scenarios.

Spectral efficiency computations are based on convolutional coding with code-rates #OTHEREFR 3 , 1 2 , 2 3 , and an oracle Adaptive Modulation and Coding (AMC) module that selects the best modulation and code-rate based on SNR.

Uplink Mimo Detection Using Ising Machines: A Multi-Stage Ising Approach

Abstract: Multiple-Input-Multiple-Output (MIMO) signal detection is central to every state-of-the-art communication system, and enhancements in error performance and computation complexity of MIMO detection would significantly enhance data rate and latency experienced by the users. Theoretically, the optimal MIMO detector is the maximum-likelihood (ML) MIMO detector; however, due to its extremely high complexity, it is not feasible for large real-world communication systems. Over the past few years, algorithms based on physics-inspired Ising solvers, like Coherent Ising machines and Quantum Annealers, have shown significant performance improvements for the MIMO detection problem. However, the current state-of-the-art is limited to low-order modulations or systems with few users. In this paper, we propose an adaptive multi-stage Ising machine-based MIMO detector that extends the performance gains of physics-inspired computation to Large and Massive MIMO systems with a large number of users and very high modulation schemes (up to 256-QAM). We enhance our previously proposed delta Ising formulation and develop a heuristic that adaptively optimizes the performance and complexity of our proposed method. We perform extensive micro-benchmarking to optimize several free parameters of the system and evaluate our methods' BER and spectral efficiency for Large and Massive MIMO systems (up to 32 users and 256 QAM modulation).

Oim: Oscillator-Based Ising Machines For Solving Combinatorial Optimisation Problems

Abstract: We present a new way to make Ising machines, i.e., using networks of coupled self-sustaining nonlinear oscillators. Our scheme is theoretically rooted in a novel result that establishes that the phase dynamics of coupled oscillator systems, under the influence of subharmonic injection locking, are governed by a Lyapunov function that is closely related to the Ising Hamiltonian of the coupling graph. As a result, the dynamics of such oscillator networks evolve naturally to local minima of the Lyapunov function. Two simple additional steps (i.e., adding noise, and turning subharmonic locking on and off smoothly) enable the network to find excellent solutions of Ising problems. We demonstrate our method on Ising versions of the MAX-CUT and graph colouring problems, showing that it improves on previously published results on several problems in the G benchmark set. Our scheme, which is amenable to realisation using many kinds of oscillators from different physical domains, is particularly well suited for CMOS IC implementation, offering significant practical advantages over previous techniques for making Ising machines. We present working hardware prototypes using CMOS electronic oscillators.


Same augmentations are performed on the training data as #REFR , namely 50% horizontal flips, random brightness, contrast, saturation, and hue jitter.

Particularly, the top two levels are not used in the depth encoder.

For the motion network, the pose decoder is adopted from #OTHEREFR .

The object motion decoder 4 uses 9 and 2 RMUs in level 4 and the remained levels, respectively.

RMUs are not shared across different levels in order to maximize filter diversity for different scales.

Training Details. The whole system is implemented in TensorFlow [1] .

Following #OTHEREFR , the length of each image sequence is fixed to 3 frames. The central frame is treated as the target view.

The depth and motion networks are jointly trained using Adam #OTHEREFR with a batch size varying from 16 to 40 on multiple GPUs.

To address the stationary pixels and the oc-clusion problem, the auto-masking and the per-pixel minimum reprojection loss #OTHEREFR are adopted.

Depth map and motion field are regularized by an edge-aware smoothness loss #OTHEREFR while the proposed outlier-aware regularization loss is further imposed on the object motion field.

The selfsupervision #OTHEREFR is also adopted but no cropping is applied. Some parts of RM-Depth require pre-training #OTHEREFR .

Rm-Depth: Unsupervised Learning Of Recurrent Monocular Depth In Dynamic Scenes *

Abstract: Unsupervised methods have showed promising results on monocular depth estimation. However, the training data must be captured in scenes without moving objects. To push the envelope of accuracy, recent methods tend to increase their model parameters. In this paper, an unsupervised learning framework is proposed to jointly predict monocular depth and complete 3D motion including the motions of moving objects and camera. (1) Recurrent modulation units are used to adaptively and iteratively fuse encoder and decoder features. This not only improves the singleimage depth inference but also does not overspend model parameters. (2) Instead of using a single set of filters for upsampling, multiple sets of filters are devised for the residual upsampling. This facilitates the learning of edgepreserving filters and leads to the improved performance. (3) A warping-based network is used to estimate a motion field of moving objects without using semantic priors. This breaks down the requirement of scene rigidity and allows to use general videos for the unsupervised learning. The motion field is further regularized by an outlier-aware training loss. Despite the depth model just uses a single image in test time and 2.97M parameters, it achieves state-of-the-art results on the KITTI and Cityscapes benchmarks. * This research work is not for commercial use unless a prior arrangement has been made with the author. 1 The words, ego-motion, camera motion and pose, are interchangeably used throughout the paper.

Digging Into Self-Supervised Monocular Depth Estimation

Abstract: Per-pixel ground-truth depth data is challenging to acquire at scale. To overcome this limitation, self-supervised learning has emerged as a promising alternative for training models to perform monocular depth estimation. In this paper, we propose a set of improvements, which together result in both quantitatively and qualitatively improved depth maps compared to competing self-supervised methods. Research on self-supervised monocular training usually explores increasingly complex architectures, loss functions, and image formation models, all of which have recently helped to close the gap with fully-supervised methods. We show that a surprisingly simple model, and associated design choices, lead to superior predictions. In particular, we propose (i) a minimum reprojection loss, designed to robustly handle occlusions, (ii) a full-resolution multi-scale sampling method that reduces visual artifacts, and (iii) an auto-masking loss to ignore training pixels that violate camera motion assumptions. We demonstrate the effectiveness of each component in isolation, and show high quality, state-of-the-art results on the KITTI benchmark.


Using MeshGAN, the first intrinsic GAN architecture operating directly on 3D meshes, we can generate high-fidelity 3D faces with rich identities, and expressions #REFR .

It samples images at various scales using a patchbased discriminator to produce radiance fields.

Another strategy is to use an effective and efficient tri-plane-based 3D GAN framework #OTHEREFR .

It uses dual discrimination to promote consistency from several viewpoints while the generator is conditioned on poses to faithfully model attribute distributions dependent on the pose in the real-world datasets.

• Generating texture models or 3DMMs: In #OTHEREFR , the authors introduce a novel 3D Morphable Model (3DMM) in the form of a GAN Texture Model to provide excellent facial shape and texture reconstructions in arbitrary recording conditions from 2D images.

They also show the results to be both photorealistic and identity preserving in both qualitative and quantitative experiments.

• Feedback learning: By considering the generator network as an encoder and decoder, the spatial output from multiple discriminators can be used to provide feedback to the generator so it can improve on its previous generations using Adaptive Spatial Transform #OTHEREFR .

• Meshes: In industrial design, gaming, computer graphics and other digital art, automatically generating shapes based on meshes is necessary.

Most of the current research is involved with voxel and point cloud generation, alienating itself from the design and graphics communities.

MeshGAN, as mentioned above, is the first intrinsic GAN architecture operating directly on 3D meshes and can generate high-fidelity 3D faces with rich identities and expressions.

To automatically generate shapes based on meshes, we use the signed distance function representation to generate detail-preserving threedimensional surface meshes #OTHEREFR .

3D Gans And Latent Space: A Comprehensive Survey

Abstract: Generative Adversarial Networks (GANs) have emerged as a significant player in generative modeling by mapping lower-dimensional random noise to higher-dimensional spaces. These networks have been used to generate high-resolution images and 3D objects. The efficient modeling of 3D objects and human faces is crucial in the development process of 3D graphical environments such as games or simulations. 3D GANs are a new type of generative model used for 3D reconstruction, point cloud reconstruction, and 3D semantic scene completion. The choice of distribution for noise is critical as it represents the latent space. Understanding a GAN's latent space is essential for fine-tuning the generated samples, as demonstrated by the morphing of semantically meaningful parts of images. In this work, we explore the latent space and 3D GANs, examine several GAN variants and training methods to gain insights into improving 3D GAN training, and suggest potential future directions for further research.

Meshgan: Non-Linear 3D Morphable Models Of Faces

Abstract: Generative Adversarial Networks (GANs) are currently the method of choice for generating visual data. Certain GAN architectures and training methods have demonstrated exceptional performance in generating realistic synthetic images (in particular, of human faces). However, for 3D object, GANs still fall short of the success they have had with images. One of the reasons is due to the fact that so far GANs have been applied as 3D convolutional architectures to discrete volumetric representations of 3D objects. In this paper, we propose the first intrinsic GANs architecture operating directly on 3D meshes (named as MeshGAN). Both quantitative and qualitative results are provided to show that MeshGAN can be used to generate high-fidelity 3D face with rich identities and expressions.


Besides, we comprehensively test the robustness of SAM with 18 types of data corruption at 5 severity levels by following #REFR .

In this study, two widely-used surgical instrument segmentation datasets, i.e., EndoVis17 #OTHEREFR and EndoVis18 #OTHEREFR , have been adopted in our experiments. Our evaluation involves three categories.

Firstly, we have provided both quantitative and qualitative assessments on the promptable segmentation performance of SAM, with bounding boxes and single points, for binary and instrument-wise segmentation.

Moreover, we also examine SAM on its automatic mask generation in unprompted settings for surgical scene segmentation.

Sam Meets Robotic Surgery: An Empirical Study In Robustness Perspective

Abstract: Segment Anything Model (SAM) is a foundation model for semantic segmentation and shows excellent generalization capability with the prompts. In this empirical study, we investigate the robustness and zero-shot generalizability of the SAM in the domain of robotic surgery in various settings of (i) prompted vs. unprompted; (ii) bounding box vs. points-based prompt; (iii) generalization under corruptions and perturbations with five severity levels; and (iv) state-of-the-art supervised model vs. SAM. We conduct all the observations with two well-known robotic instrument segmentation datasets of MICCAI EndoVis 2017 and 2018 challenges. Our extensive evaluation results reveal that although SAM shows remarkable zero-shot generalization ability with bounding box prompts, it struggles to segment the whole instrument with point-based prompts and unprompted settings. Furthermore, our qualitative figures demonstrate that the model either failed to predict the parts of the instrument mask (e.g., jaws, wrist) or predicted parts of the instrument as different classes in the scenario of overlapping instruments within the same bounding box or with the point-based prompt. In fact, it is unable to identify instruments in some complex surgical scenarios of blood, reflection, blur, and shade. Additionally, SAM is insufficiently robust to maintain high performance when subjected to various forms of data corruption. Therefore, we can argue that SAM is not ready for downstream surgical tasks without further domain-specific fine-tuning.

Benchmarking Neural Network Robustness To Common Corruptions And Surface Variations

Abstract: In this paper we establish rigorous benchmarks for image classifier robustness. Our first benchmark, ImageNet-C, standardizes and expands the corruption robustness topic, while showing which classifiers are preferable in safety-critical applications. Then we propose a new dataset called ImageNet-P which enables researchers to benchmark a classifier's robustness to common perturbations. Unlike recent robustness research, this benchmark evaluates performance on common corruptions and perturbations not worst-case adversarial perturbations. We find that there are negligible changes in relative corruption robustness from AlexNet classifiers to ResNet classifiers. Afterward we discover ways to enhance corruption and perturbation robustness. We even find that a bypassed adversarial defense provides substantial common perturbation robustness. Together our benchmarks may aid future work toward networks that robustly generalize.


The ground-truth graph is a large network with 70 nodes and 123 edges which is available in the bnlearn #REFR ) repository.

(1989) ) designed to provide an alarm message for patients, and has an associated synthetic dataset.

In particular, it implements a cautionary alarm message for patient monitoring.

The ground-truth graph is a medium-sized network with 37 nodes and 46 edges. This dataset was used by #OTHEREFR , and to evaluate their approaches. The ground-truth network is available in this repository: https://www.bnlearn.com/bnrepository/.

• HEPAR2: It is a probabilistic causal model for the diagnosis of liver disorders #OTHEREFR ).

This causal Bayesian network tries to capture the causal links among different risk factors, diseases, symptoms, and test results.

A Survey On Causal Discovery Methods For Temporal And Non-Temporal Data

Abstract: Causal Discovery (CD) is the process of identifying the cause-effect relationships among the variables of a system from data. Over the years, several methods have been developed primarily based on the statistical properties of data to uncover the underlying causal mechanism. In this study, we present an extensive discussion on the methods designed to perform causal discovery from both independent and identically distributed (i.i.d.) data and time series data. For this purpose, we first introduce the common terminologies in causal discovery, and then provide a comprehensive discussion of the algorithms designed to identify the causal edges in different settings. We further discuss some of the benchmark datasets available for evaluating the performance of the causal discovery methods, available tools or software packages to perform causal discovery readily, and the common metrics used to evaluate these methods. We also test some common causal discovery algorithms on different benchmark datasets, and compare their performances. Finally, we conclude by presenting the common challenges involved in causal discovery, and also, discuss the applications of causal discovery in multiple areas of interest.

Learning Bayesian Networks With The Bnlearn R Package

Abstract: bnlearn is an R package (R Development Core Team 2010) which includes several algorithms for learning the structure of Bayesian networks with either discrete or continuous variables. Both constraint-based and score-based algorithms are implemented, and can use the functionality provided by the snow package (Tierney et al. 2008) to improve their performance via parallel computing. Several network scores and conditional independence algorithms are available for both the learning algorithms and independent use. Advanced plotting options are provided by the Rgraphviz package (Gentry et al. 2010).


In a seminal work, in 2015, Mahendran and Vedaldi #REFR set out to analyse the visual information contained in both shallow (e.g.

HOG) and deep feature representations, to investigate the question: given an encoding of an image, to which extent is it possible to reconstruct the image itself.

They propose an optimisation method to invert representations using gradient descent.

Among their findings are that networks retain rich information even at deep levels and that a progressively more invariant and abstract notion of the image content is formed in the network.

As face identity descriptors usually are made up of the final layer of a deep network, it would therefore be the most invariant and abstract representation.

Few works so far have investigated the inversion of a face descriptor back to a face image. Genova et al.

Id2Image: Leakage Of Non-Id Information Into Face Descriptors And Inversion From Descriptors To Images

Abstract: Embedding a face image to a descriptor vector using a deep CNN is a widely used technique in face recognition. Via several possible training strategies, such embeddings are supposed to capture only identity information. Information about the environment (such as background and lighting) or changeable aspects of the face (such as pose, expression, presence of glasses, hat etc.) should be discarded since they are not useful for recognition. In this paper, we present a surprising result that this is not the case. We show that non-ID attributes, as well as landmark positions and the image histogram can be recovered from the ID embedding of state-of-the-art face embedding networks (VGGFace2 and ArcFace). In fact, these non-ID attributes can be predicted from ID embeddings with similar accuracy to a prediction from the original image. Going further, we present an optimisation strategy that uses a generative model (specifically StyleGAN2 for faces) to recover images from an ID embedding. We show photorealistic inversion from ID embedding to face image in which not only is the ID realistically reconstructed but the pose, lighting and background/apparel to some extent as well.

Understanding Deep Image Representations By Inverting Them

Abstract: Image representations, from SIFT and Bag of Visual Words to Convolutional Neural Networks (CNNs), are a crucial component of almost any image understanding system. Nevertheless, our understanding of them remains limited. In this paper we conduct a direct analysis of the visual information contained in representations by asking the following question: given an encoding of an image, to which extent is it possible to reconstruct the image itself? To answer this question we contribute a general framework to invert representations. We show that this method can invert representations such as HOG more accurately than recent alternatives while being applicable to CNNs too. We then use this technique to study the inverse of recent state-of-the-art CNN image representations for the first time. Among our findings, we show that several layers in CNNs retain photographically accurate information about the image, with different degrees of geometric and photometric invariance.


For the identification task, CoReFace achieves competitive performance which is only 0.02% lower compared to the highest one Curricular-Face #REFR .

When there is a higher FAR bound (10 −4 ) or the evaluation dataset is larger, CoReFace still outperforms the competitors.

Finally, we demonstrate the efficacy of our method on the MegaFace Challenge.

The gallery set of MegaFace contains 1M images of 690K subjects, and the probe set is FaceScrub, which contains 100K photos of 530 unique individuals.

We follow #OTHEREFR to remove the face images with wrong labels and evaluate our method on the refined dataset. Table 3 shows the performance of different methods.

For the verification task, CoReFace outperforms all the other approaches with a clear margin.

The Broad-Face #OTHEREFR also shows competitive performance by building a dynamic queue to gain extra training on the classification layer.

Without complex structure reformation, CoReFace adds an image-image regularization to improve the feature distribution and boost the performance of open-set face recognition.

Coreface: Sample-Guided Contrastive Regularization For Deep Face Recognition

Abstract: The discriminability of feature representation is the key to open-set face recognition. Previous methods rely on the learnable weights of the classification layer that represent the identities. However, the evaluation process learns no identity representation and drops the classifier from training. This inconsistency could confuse the feature encoder in understanding the evaluation goal and hinder the effect of identity-based methods. To alleviate the above problem, we propose a novel approach namely Contrastive Regularization for Face recognition (CoReFace) to apply image-level regularization in feature representation learning. Specifically, we employ sample-guided contrastive learning to regularize the training with the image-image relationship directly, which is consistent with the evaluation process. To integrate contrastive learning into face recognition, we augment embeddings instead of images to avoid the image quality degradation. Then, we propose a novel contrastive loss for the representation distribution by incorporating an adaptive margin and a supervised contrastive mask to generate steady loss values and avoid the collision with the classification supervision signal. Finally, we discover and solve the semantically repetitive signal problem in contrastive learning by exploring new pair coupling protocols. Extensive experiments demonstrate the efficacy and efficiency of our CoReFace which is highly competitive with the state-of-the-art approaches.

Curricularface: Adaptive Curriculum Learning Loss For Deep Face Recognition

Abstract: As an emerging topic in face recognition, designing margin-based loss functions can increase the feature margin between different classes for enhanced discriminability. More recently, the idea of mining-based strategies is adopted to emphasize the misclassified samples, achieving promising results. However, during the entire training process, the prior methods either do not explicitly emphasize the sample based on its importance that renders the hard samples not fully exploited; or explicitly emphasize the effects of semi-hard/hard samples even at the early training stage that may lead to convergence issue. In this work, we propose a novel Adaptive Curriculum Learning loss (CurricularFace) that embeds the idea of curriculum learning into the loss function to achieve a novel training strategy for deep face recognition, which mainly addresses easy samples in the early training stage and hard ones in the later stage. Specifically, our CurricularFace adaptively adjusts the relative importance of easy and hard samples during different training stages. In each stage, different samples are assigned with different importance according to their corresponding difficultness. Extensive experimental results on popular benchmarks demonstrate the superiority of our CurricularFace over the state-of-the-art competitors.


Our perceptual loss(L perc in algorithm1) is defined as a l 1 loss in feature space, see equation #REFR .

To ensure the perceptual fidelity of the SR, we propose to use a perceptual loss function (L perc in algorithm1) to constrain the quality of SR results in feature space.

The perceptual loss was first introduced in #OTHEREFR , where a pretrained VGG-19 network #OTHEREFR was used to extract features.

However, our implementation differs in that we do not use the pretrained VGG-19 network, but an actively updating ResNet10 network.

More details are explained in the "Feature Matching" section below.

Here I i,j,k andÎ i,j,k represent HR and SR intensity value at the i, j, k-th voxel in all three image dimensions(W, H, D), F(•) represent the feature network.

L perc = 1 W × H × D W,H,D i,j,k=1 |F(I i,j,k ) − F(Î i,j,k )| (1)

As shown in #OTHEREFR , perceptual loss functions generally outperform MSE loss functions, because they better preserve the detailed texture of the image.

A Three-Player Gan For Super-Resolution In Magnetic Resonance Imaging

Abstract: Learning based single image super resolution (SISR) task is well investigated in 2D images. However, SISR for 3D Magnetics Resonance Images (MRI) is more challenging compared to 2D, mainly due to the increased number of neural network parameters, the larger memory requirement and the limited amount of available training data. Current SISR methods for 3D volumetric images are based on Generative Adversarial Networks (GANs), especially Wasserstein GANs due to their training stability. Other common architectures in the 2D domain, e.g. transformer models, require large amounts of training data and are therefore not suitable for the limited 3D data. However, Wasserstein GANs can be problematic because they may not converge to a global optimum and thus produce blurry results. Here, we propose a new method for 3D SR based on the GAN framework. Specifically, we use instance noise to balance the GAN training. Furthermore, we use a relativistic GAN loss function and an updating feature extractor during the training process. We show that our method produces highly accurate results. We also show that we need very few training samples. In particular, we need less than 30 samples instead of thousands of training samples that are typically required in previous studies. Finally, we show improved out-of-sample results produced by our model.

The Mechanics Of N-Player Differentiable Games

Abstract: The cornerstone underpinning deep learning is the guarantee that gradient descent on an objective converges to local minima. Unfortunately, this guarantee fails in settings, such as generative adversarial nets, where there are multiple interacting losses. The behavior of gradient-based methods in games is not well understood -- and is becoming increasingly important as adversarial and multi-objective architectures proliferate. In this paper, we develop new techniques to understand and control the dynamics in general games. The key result is to decompose the second-order dynamics into two components. The first is related to potential games, which reduce to gradient descent on an implicit function; the second relates to Hamiltonian games, a new class of games that obey a conservation law, akin to conservation laws in classical mechanical systems. The decomposition motivates Symplectic Gradient Adjustment (SGA), a new algorithm for finding stable fixed points in general games. Basic experiments show SGA is competitive with recently proposed algorithms for finding stable fixed points in GANs -- whilst at the same time being applicable to -- and having guarantees in -- much more general games.


We also see sub-optimal results for Union #REFR due to errors in the approximation of unweighted union, similar to object detection experiments.

a small λ value on target domain image without any pseudo labels.

This strategy is similar to our noise mixing during the warmup, but is used throughout the training.

Note that this approach performs worse than our AT basedline.

This is because although noise mixing could be helpful in general, as shown by both AFAN #OTHEREFR and our following ablation studies, heavily relying on it in the adaptation phase of Mean Teacher can lead to bias towards the source domain due to the fact that "mixed-in" target information is only limited to a tiny amount to act as a domain-aware augmentation.

Indeed, we believe for cross-domain mean teacher, the pseudo labels are much stronger target signals and should be taken advantage of appropriately.

PASCAL VOC → Watercolor2k Next, we are interested in answering the question whether or not the encouraging gains observed in PASCAL VOC → Clipart1k can be reproduced on a different dataset.

To do this, we use Wa-tercolor2k and evaluate the performance of PASCAL VOC → Watercolor2k adaptation.

Note that after experimenting with Clipart1k, we narrowed down our set of hyperparameters to ones that work best for both Adaptive Teacher and our method for fair competition.

For Watercolor2k, to test our method's robustness, we directly perform grid search on this small set of hyper-parameters without any further tuning or manual supervision.

Nonetheless, even without exhaustive tuning, our results in Table 5 show that we can still outperform AT (mAP=57.7) with a +1.5 improvement and archive mAP=59.3.

Lossmix: Simplify And Generalize Mixup For Object Detection And Beyond

Abstract: The success of data mixing augmentations in image classification tasks has been well-received. However, these techniques cannot be readily applied to object detection due to challenges such as spatial misalignment, foreground/background distinction, and plurality of instances. To tackle these issues, we first introduce a novel conceptual framework called Supervision Interpolation, which offers a fresh perspective on interpolation-based augmentations by relaxing and generalizing Mixup. Building on this framework, we propose LossMix, a simple yet versatile and effective regularization that enhances the performance and robustness of object detectors and more. Our key insight is that we can effectively regularize the training on mixed data by interpolating their loss errors instead of ground truth labels. Empirical results on the PASCAL VOC and MS COCO datasets demonstrate that LossMix consistently outperforms currently popular mixing strategies. Furthermore, we design a two-stage domain mixing method that leverages LossMix to surpass Adaptive Teacher (CVPR 2022) and set a new state of the art for unsupervised domain adaptation. * Work done during a residency with Mineral.

Bag Of Freebies For Training Object Detection Neural Networks

Abstract: Training heuristics greatly improve various image classification model accuracies~\cite{he2018bag}. Object detection models, however, have more complex neural network structures and optimization targets. The training strategies and pipelines dramatically vary among different models. In this works, we explore training tweaks that apply to various models including Faster R-CNN and YOLOv3. These tweaks do not change the model architectures, therefore, the inference costs remain the same. Our empirical results demonstrate that, however, these freebies can improve up to 5% absolute precision compared to state-of-the-art baselines.


To complete the proof, we need to show that any optimal solution to (28) satisfies the dropped constraint ψ * (Q) ≤ 0 and is therefore also a solution to the original convex program #REFR .

n i=1 g −1 (q k,i ) = 1 for all k ∈ [K].

We now consider the relaxed convex program Q ∈ arg min

Q∈R Kn 1 2 Q CXX CQ F (Q) (28) subject to − q k,i < 0 for all i ∈ [n] and k ∈ [K], and 1 − n i=1 g −1 q k,i ≤ 0 for all k ∈ [K]

that is obtained by dropping the constraint ψ * (Q) ≤ 0 and relaxing the equality constraint 1−

n i=1 g −1 q k,i = 0 for all k ∈ [K] to an inequality constraint, 1 − n i=1 g −1 q k,i ≤ 0.

(Note that by Lemma 15, the equality constraint 1 − n i=1 g −1 q k,i = 0 for all k ∈ [K], also ensures that Q ∈ dom ψ * .) It is necessary and sufficient for any optimal solutionQ to the relaxed convex program #OTHEREFR to satisfy its KKT conditions, listed below:

First, we claim that any optimal solutionQ needs to satisfy 1 −

General Loss Functions Lead To (Approximate) Interpolation In High Dimensions

Abstract: We provide a unified framework, applicable to a general family of convex losses and across binary and multiclass settings in the overparameterized regime, to approximately characterize the implicit bias of gradient descent in closed form. Specifically, we show that the implicit bias is approximated (but not exactly equal to) the minimum-norm interpolation in high dimensions, which arises from training on the squared loss. In contrast to prior work which was tailored to exponentially-tailed losses and used the intermediate support-vector-machine formulation, our framework directly builds on the primaldual analysis of [29], allowing us to provide new approximate equivalences for general convex losses through a novel sensitivity analysis. Our framework also recovers existing exact equivalence results for exponentially-tailed losses across binary and multiclass settings. Finally, we provide evidence for the tightness of our techniques, which we use to demonstrate the effect of certain loss functions designed for out-of-distribution problems on the closed-form solution.

Learning Imbalanced Datasets With Label-Distribution-Aware Margin Loss

Abstract: Deep learning algorithms can fare poorly when the training dataset suffers from heavy class-imbalance but the testing criterion requires good generalization on less frequent classes. We design two novel methods to improve performance in such scenarios. First, we propose a theoretically-principled label-distribution-aware margin (LDAM) loss motivated by minimizing a margin-based generalization bound. This loss replaces the standard cross-entropy objective during training and can be applied with prior strategies for training with class-imbalance such as re-weighting or re-sampling. Second, we propose a simple, yet effective, training schedule that defers re-weighting until after the initial stage, allowing the model to learn an initial representation while avoiding some of the complications associated with re-weighting or re-sampling. We test our methods on several benchmark vision tasks including the real-world imbalanced dataset iNaturalist 2018. Our experiments show that either of these methods alone can already improve over existing techniques and their combination achieves even better performance gains.

original convex program
dropped constraint ψ


We first demonstrate the ability of our NVF to reconstruct non-watertight meshes by category-specific reconstruction in Sec. #REFR Table 1 . Quantitative evaluation on ShapeNet Cars.

We evaluate the effectiveness of our framework on four tasks: 1) category-specific, 2) category-agnostic, 3) category-unseen and 4) cross-domain reconstruction.

We train and evaluate our method on the raw data of the ShapeNet "Car" category.

Our method achieves better performance than the state-ofthe-art UDF-based methods.

we compare our NVF with existing methods on categoryagnostic and category-unseen reconstruction in Sec. 4.3.

We also test cross-domain reconstruction by reconstructing real scanned data without training or fine-tuning in Sec. 4.4. Implementations.

We employ PointTransformer #OTHEREFR as our feature encoder and set k = 16 for the nearest points.

Neural Vector Fields: Implicit Representation By Explicit Learning

Abstract: Deep neural networks (DNNs) are widely applied for nowadays 3D surface reconstruction tasks and such methods can be further divided into two categories, which respectively warp templates explicitly by moving vertices or represent 3D surfaces implicitly as signed or unsigned distance functions. Taking advantage of both advanced explicit learning process and powerful representation ability of implicit functions, we propose a novel 3D representation method, Neural Vector Fields (NVF). It not only adopts the explicit learning process to manipulate meshes directly, but also leverages the implicit representation of unsigned distance functions (UDFs) to break the barriers in resolution and topology. Specifically, our method first predicts the displacements from queries towards the surface and models the shapes as Vector Fields. Rather than relying on network differentiation to obtain direction fields as most existing UDF-based methods, the produced vector fields encode the distance and direction fields both and mitigate the ambiguity at "ridge" points, such that the calculation of direction fields is straightforward and differentiation-free. The differentiation-free characteristic enables us to further learn a shape codebook via Vector Quantization, which encodes the cross-object priors, accelerates the training procedure, and boosts model generalization on cross-category reconstruction. The extensive experiments on surface reconstruction benchmarks indicate that our method outperforms those state-of-the-art methods in different evaluation scenarios including watertight vs nonwatertight shapes, category-specific vs category-agnostic reconstruction, category-unseen reconstruction, and crossdomain reconstruction. Our code will be publicly released.

Shape Reconstruction By Learning Differentiable Surface Representations

Abstract: Generative models that produce point clouds have emerged as a powerful tool to represent 3D surfaces, and the best current ones rely on learning an ensemble of parametric representations. Unfortunately, they offer no control over the deformations of the surface patches that form the ensemble and thus fail to prevent them from either overlapping or collapsing into single points or lines. As a consequence, computing shape properties such as surface normals and curvatures becomes difficult and unreliable. ::: In this paper, we show that we can exploit the inherent differentiability of deep networks to leverage differential surface properties during training so as to prevent patch collapse and strongly reduce patch overlap. Furthermore, this lets us reliably compute quantities such as surface normals and curvatures. We will demonstrate on several tasks that this yields more accurate surface reconstructions than the state-of-the-art methods in terms of normals estimation and amount of collapsed and overlapped patches.


FastText #REFR conceives each word as a group of n-grams instead of a single string. It is trained to vectorize n-grams.

It employs a local context window, as a continuous bagof-word (order-agnostic) or a continuous skip-gram (order-aware).

The latter can link words that behave similarly in a sentence, but fails to utilize the statistics of a corpus.

GloVe #OTHEREFR combines matrix factorization, i.e., the global cooccurrence counts, with a local context window, i.e., word analogy.

It is trained on large corpora, such as Wikipedia, to provide pre-trained vectors for general use.

Since it operates on a global dictionary, it identifies words with a specific writing and fails to detect slight modifications.

It then represents each word as the sum of its underlying n-grams.

Pre-Trained Embeddings For Entity Resolution: An Experimental Analysis [Experiment, Analysis & Benchmark

Abstract: Many recent works on Entity Resolution (ER) leverage Deep Learning techniques involving language models to improve effectiveness. This is applied to both main steps of ER, i.e., blocking and matching. Several pre-trained embeddings have been tested, with the most popular ones being fastText and variants of the BERT model. However, there is no detailed analysis of their pros and cons. To cover this gap, we perform a thorough experimental analysis of 12 popular language models over 17 established benchmark datasets. First, we assess their vectorization overhead for converting all input entities into dense embeddings vectors. Second, we investigate their blocking performance, performing a detailed scalability analysis, and comparing them with the state-of-the-art deep learning-based blocking method. Third, we conclude with their relative performance for both supervised and unsupervised matching. Our experimental results provide novel insights into the strengths and weaknesses of the main language models, facilitating researchers and practitioners to select the most suitable ones in practice.

Enriching Word Vectors With Subword Information

Abstract: Continuous word representations, trained on large unlabeled corpora are useful for many natural language processing tasks. Popular models to learn such representations ignore the morphology of words, by assigning a distinct vector to each word. This is a limitation, especially for languages with large vocabularies and many rare words. In this paper, we propose a new approach based on the skipgram model, where each word is represented as a bag of character n-grams. A vector representation is associated to each character n-gram, words being represented as the sum of these representations. Our method is fast, allowing to train models on large corpora quickly and allows to compute word representations for words that did not appear in the training data. We evaluate our word representations on nine different languages, both on word similarity and analogy tasks. By comparing to recently proposed morphological word representations, we show that our vectors achieve state-of-the-art performance on these tasks.


Like all models derived from T5 models, #REFR , we pose AMR parsing as a text-totext problem and train models to transfer a text to a linearized AMR graph with the task prefix amr generation.

2 AMR Parsing with FLAN-T5 Models Flan-T5 models, (Chung et al., 2022) , are obtained by instruction fine-tuning T5-LM adapted models, , on a collection of 1.8K instruction annotated tasks.

1 They are prefix language models 2 and achieve strong few-shot performance even compared to much larger models, such as PaLM 62B.

FLAN-T5 model size variants, all of which use 32,128 unique vocabulary, is shown in Table 1 .

Amr Parsing With Instruction Fine-Tuned Pre-Trained Language Models

Abstract: Instruction fine-tuned language models on a collection of instruction annotated datasets (FLAN) have shown highly effective to improve model performance and generalization to unseen tasks. However, a majority of standard parsing tasks including abstract meaning representation (AMR), universal dependency (UD), semantic role labeling (SRL) has been excluded from the FLAN collections for both model training and evaluations. In this paper, we take one of such instruction fine-tuned pre-trained language models, i.e. FLAN-T5, and fine-tune them for AMR parsing. Our extensive experiments on various AMR parsing tasks including AMR2.0, AMR3.0 and BioAMR indicate that FLAN-T5 fine-tuned models out-perform previous state-of-the-art models across all tasks. In addition, full finetuning followed by the parameter efficient finetuning, LoRA, further improves the model performances, setting new state-of-the-arts in Smatch on AMR2.0 (86.4), AMR3.0 (84.9) and BioAMR (82.3).

Exploring The Limits Of Transfer Learning With A Unified Text-To-Text Transformer

Abstract: Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new "Colossal Clean Crawled Corpus", we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.