UCF-Crime #REFR contains 1,900 untrimmed videos with 13 different types of anomalies, among which 1,610 videos are for training and 290 videos are for the test.

We evaluate the proposed method on three public datasets: ShanghaiTech #OTHEREFR , #OTHEREFR , UCF-Crime #OTHEREFR , and UBnormal #OTHEREFR .

ShanghaiTech #OTHEREFR consists of 437 videos with 130 abnormal events in 13 scenes. In the original dataset, all training videos are normal.

#OTHEREFR reorganized the dataset by selecting some abnormal testing videos into the training set for WS-VAD. We follow this new split way in our experiments.

UBnormal #OTHEREFR is a new large-scale dataset synthesized by the Cinema4D software.

It consists of 543 videos with 22 types of anomalies on 29 virtual scenes, including 268 videos for training, 64 videos for validation, and 211 videos for test.

Since it has a higher number of anomaly types occurring across a larger set of scenes, this dataset is more challenging for anomaly detection.

Following the common practice #OTHEREFR , #OTHEREFR , we take the area under the curve (AUC) of the framelevel receiver operating characteristics (ROC) for evaluation. A higher AUC indicates better performance.

Long-Short Temporal Co-Teaching For Weakly Supervised Video Anomaly Detection

Abstract: Weakly supervised video anomaly detection (WS-VAD) is a challenging problem that aims to learn VAD models only with video-level annotations. In this work, we propose a Long-Short Temporal Co-teaching (LSTC) method to address the WS-VAD problem. It constructs two tubelet-based spatiotemporal transformer networks to learn from short-and longterm video clips respectively. Each network is trained with respect to a multiple instance learning (MIL)-based ranking loss, together with a cross-entropy loss when clip-level pseudo labels are available. A co-teaching strategy is adopted to train the two networks. That is, clip-level pseudo labels generated from each network are used to supervise the other one at the next training round, and the two networks are learned alternatively and iteratively. Our proposed method is able to better deal with the anomalies with varying durations as well as subtle anomalies. Extensive experiments on three public datasets demonstrate that our method outperforms state-of-the-art WS-VAD methods.

Real-World Anomaly Detection In Surveillance Videos

Abstract: Surveillance videos are able to capture a variety of realistic anomalies. In this paper, we propose to learn anomalies by exploiting both normal and anomalous videos. To avoid annotating the anomalous segments or clips in training videos, which is very time consuming, we propose to learn anomaly through the deep multiple instance ranking framework by leveraging weakly labeled training videos, i.e. the training labels (anomalous or normal) are at video-level instead of clip-level. In our approach, we consider normal and anomalous videos as bags and video segments as instances in multiple instance learning (MIL), and automatically learn a deep anomaly ranking model that predicts high anomaly scores for anomalous video segments. Furthermore, we introduce sparsity and temporal smoothness constraints in the ranking loss function to better localize anomaly during training. We also introduce a new large-scale first of its kind dataset of 128 hours of videos. It consists of 1900 long and untrimmed real-world surveillance videos, with 13 realistic anomalies such as fighting, road accident, burglary, robbery, etc. as well as normal activities. This dataset can be used for two tasks. First, general anomaly detection considering all anomalies in one group and all normal activities in another group. Second, for recognizing each of 13 anomalous activities. Our experimental results show that our MIL method for anomaly detection achieves significant improvement on anomaly detection performance as compared to the state-of-the-art approaches. We provide the results of several recent deep learning baselines on anomalous activity recognition. The low recognition performance of these baselines reveals that our dataset is very challenging and opens more opportunities for future work.


These adversarial attacks have also been successfully transferred between models designed to perform similar tasks, even if their architectures or training sets are different #REFR .

One example is the reconstruction of the faces used to train a facial recognition system #OTHEREFR .

Model extraction exploratory attacks seek to use probing to construct a replica of the victim model.

The replica can then be used to stage further attacks.

In #OTHEREFR , the authors demonstrate that they can accomplish the same black-box model inversion attack performed in #OTHEREFR with fewer queries by extracting the victim model and then performing a white-box inversion attack on the replica.

In summary, evasion, poisoning, backdoor, and exploratory attacks (illustrated in Fig. 1 ) are major challenges for ML.

This property dramatically complicates the challenge of defending ML models deployed in embedded systems against adversarial attacks, enabling scenarios in which exploratory attacks are used to extract the victim model and generate a surrogate model against which an evasion, poisoning, or backdoor attack can be crafted and then transferred back to the victim model.

Adversarial Attacks On Machine Learning In Embedded And Iot Platforms

Abstract: Machine learning (ML) algorithms are increasingly being integrated into embedded and IoT systems that surround us, and they are vulnerable to adversarial attacks. The deployment of these ML algorithms on resource-limited embedded platforms also requires the use of model compression techniques. The impact of such model compression techniques on adversarial robustness in ML is an important and emerging area of research. This article provides an overview of the landscape of adversarial attacks and ML model compression techniques relevant to embedded systems. We then describe efforts that seek to understand the relationship between adversarial attacks and ML model compression before discussing open problems in this area.

Transferability In Machine Learning: From Phenomena To Black-Box Attacks Using Adversarial Samples

Abstract: Many machine learning models are vulnerable to adversarial examples: inputs that are specially crafted to cause a machine learning model to produce an incorrect output. Adversarial examples that affect one model often affect another model, even if the two models have different architectures or were trained on different training sets, so long as both models were trained to perform the same task. An attacker may therefore train their own substitute model, craft adversarial examples against the substitute, and transfer them to a victim model, with very little information about the victim. Recent work has further developed a technique that uses the victim model as an oracle to label a synthetic training set for the substitute, so the attacker need not even collect a training set to mount the attack. We extend these recent techniques using reservoir sampling to greatly enhance the efficiency of the training procedure for the substitute model. We introduce new transferability attacks between previously unexplored (substitute, victim) pairs of machine learning model classes, most notably SVMs and decision trees. We demonstrate our attacks on two commercial machine learning classification systems from Amazon (96.19% misclassification rate) and Google (88.94%) using only 800 queries of the victim model, thereby showing that existing machine learning approaches are in general vulnerable to systematic black-box attacks regardless of their structure.


The representation due to a perfect alignment is called a realization of the framework #REFR .

The consensus representation of the framework Θ due to an alignment S is given by Θ(S) := H * (:, 1 :

n) = S T BL † Γ (:, 1 : n) I n − n −1 1 n 1 T n .

Non-Degenerate Rigid Alignment In A Patch Framework

Abstract: Given a set of overlapping local views (patches) of a dataset, we consider the problem of finding a rigid alignment of the views that minimizes a 2-norm based alignment error. In general, the views are noisy and a perfect alignment may not exist. In this work, we characterize the non-degeneracy of an alignment in the noisy setting based on the kernel and positivity of a certain matrix. This leads to a polynomial time algorithm for testing the non-degeneracy of a given alignment. Consequently, we focus on Riemannian gradient descent for minimization of the error and obtain a sufficient condition on an alignment for the algorithm to converge (locally) linearly to it. In the case of noiseless views, a perfect alignment exists, resulting in a realization of the points that respects the geometry of the views. Under a mild condition on the views, we show that the non-degeneracy of a perfect alignment is equivalent to the local rigidity of the resulting realization. By specializing the characterization of a non-degenerate alignment to the noiseless setting, we obtain necessary and sufficient conditions on the overlapping structure of the views for a locally rigid realization. Similar results are also obtained in the context of global rigidity.

On Affine Rigidity

Abstract: We define the notion of affine rigidity of a hypergraph and prove a variety of fundamental results for this notion. First, we show that affine rigidity can be determined by the rank of a specific matrix which implies that affine rigidity is a generic property of the hypergraph.Then we prove that if a graph is is $(d+1)$-vertex-connected, then it must be "generically neighborhood affinely rigid" in $d$-dimensional space. This implies that if a graph is $(d+1)$-vertex-connected then any generic framework of its squared graph must be universally rigid. ::: Our results, and affine rigidity more generally, have natural applications in point registration and localization, as well as connections to manifold learning.


We acquire video tracks of the original recordings, and align processed video data with the audio one to form audio-visual data, similar to LRS3-TED #REFR .

To improve matching recall on the train set, we develop a fuzzy text matching strategy: we first segment TED2020 source and target sentences by punctuation if both sides of the sentence pair contain the same amount of segments.

Then we normalize TED2020 and LRS3-TED texts by punctuation removal and lowercasing.

Finally, we conduct exact text matching between the two corpora.

For LRS3-TED train set examples without a match from TED2020, we acquire pseudotranslation labels from a machine translation model, M2M-100 418M #OTHEREFR with default decoding hyper-parameters.

For non-English talks, we reuse the audio-only data, transcriptions and text translations collected by mTEDx #OTHEREFR . Our data split also follows mTEDx.

Although all the audio data in mTEDx is transcribed, only a subset of it is translated.

We acquire pseudo-translation labels from M2M-100 418M #OTHEREFR for the untranslated train set examples with default decoding hyper-parameters.

In Table 1 , we provide duration and speaker statistics for each data split in MuAViC.

MuAViC covers 9 languages providing total 1200 hours of transcribed audio-visual data from more than 8000 speakers (number of speakers estimated based on TED/TEDx talk IDs).

It also provides text translations into or out of English for 6 of the 9 languages.

Muavic: A Multilingual Audio-Visual Corpus For Robust Speech Recognition And Robust Speech-To-Text Translation

Abstract: We introduce MuAViC, a multilingual audiovisual corpus for robust speech recognition and robust speech-to-text translation providing 1200 hours of audiovisual speech in 9 languages. It is fully transcribed and covers 6 English-to-X translation as well as 6 X-to-English translation directions. To the best of our knowledge, this is the first open benchmark for audiovisual speech-to-text translation and the largest open benchmark for multilingual audiovisual speech recognition. Our baseline results show that MuAViC is effective for building noise-robust speech recognition and translation models. We make the corpus available at https://github.com/ facebookresearch/muavic.

Lrs3-Ted: A Large-Scale Dataset For Visual Speech Recognition

Abstract: This paper introduces a new multi-modal dataset for visual and audio-visual speech recognition. It includes face tracks from over 400 hours of TED and TEDx videos, along with the corresponding subtitles and word alignment boundaries. The new dataset is substantially larger in scale compared to other public datasets that are available for general research.


Nonetheless, the lack of a perfect weight calibration can be circumvented via inthe-loop training on the BrainScaleS-1 system, as shown for inference tasks in previous results #REFR .

12d ; this renders the search for a more precise fit function unbeneficial.

In addition, the per wafer calibration opted over a per neuron circuit calibration introduces a dominant error due to the deviations between neuron circuits, shown in fig. 12e .

A precise weight calibration within a reasonable runtime would be achievable via a parallel measurement of each neuron circuit.

This would also allow to exclude neurons showing unintended behavior.

However, this is not possible with the currently used analog readout system.

3) Calibration Based Exclusion of Components: The operation of the HICANNs during the calibration is similar to the operation during experiments.

All components have to work correctly for the calibration to succeed. Failing calibrations indicate unintended behavior.

This allows for testing the whole die, especially the analog circuits that cannot be tested directly. Additionally, thresholds can be defined to exclude outliers.

Consequently, neurons that do not pass all calibration steps are excluded from the availability database.

Numbers of calibration based excluded neurons on a typical wafer are given in table II.

From Clean Room To Machine Room: Commissioning Of The First-Generation Brainscales Wafer-Scale Neuromorphic System

Abstract: The first-generation of BrainScaleS, also referred to as BrainScaleS-1, is a neuromorphic system for emulating large-scale networks of spiking neurons. Following a "physical modeling" principle, its VLSI circuits are designed to emulate the dynamics of biological examples: analog circuits implement neurons and synapses with time constants that arise from their electronic components' intrinsic properties. It operates in continuous time, with dynamics typically matching an acceleration factor of 10 000 compared to the biological regime. A fault-tolerant design allows it to achieve wafer-scale integration despite unavoidable analog variability and component failures. In this paper, we present the commissioning process of a BrainScaleS-1 wafer module, providing a short description of the system's physical components, illustrating the steps taken during its assembly and the measures taken to operate it. Furthermore, we reflect on the system's development process and the lessons learned to conclude with a demonstration of its functionality by emulating a wafer-scale synchronous firing chain, the largest spiking network emulation ran with analog components and individual synapses to date. Index Terms-Neuromorphic hardware, wafer-scale integration, spiking neural networks, emulated networks, analog neuromorphic devices, synfire chains.

Neuromorphic Hardware In The Loop: Training A Deep Spiking Network On The Brainscales Wafer-Scale System

Abstract: Emulating spiking neural networks on analog neuromorphic hardware offers several advantages over simulating them on conventional computers, particularly in terms of speed and energy consumption. However, this usually comes at the cost of reduced control over the dynamics of the emulated networks. In this paper, we demonstrate how iterative training of a hardware-emulated network can compensate for anomalies induced by the analog substrate. We first convert a deep neural network trained in software to a spiking network on the BrainScaleS wafer-scale neuromorphic system, thereby enabling an acceleration factor of 10000 compared to the biological time domain. This mapping is followed by the in-the-loop training, where in each training step, the network activity is first recorded in hardware and then used to compute the parameter updates in software via backpropagation. An essential finding is that the parameter updates do not have to be precise, but only need to approximately follow the correct gradient, which simplifies the computation of updates. Using this approach, after only several tens of iterations, the spiking network shows an accuracy close to the ideal software-emulated prototype. The presented techniques show that deep spiking networks emulated on analog neuromorphic devices can attain good computational performance despite the inherent variations of the analog substrate.

BrainScaleS-1 system
inthe-loop training


Realization Of Slack Federated Adversarial Training

However, some relaxation on the assumptions could be further improved by considering more practical SGD theories #REFR , which is beyond the scope of this paper and we leave in the future works.

In this case, the increment of the first term of B can be totally counteracted by the loss of the second term of B so that in sum B becomes smaller.

Then, we can have a tighter upper bound for SFAT in Eq. (4) to achieve a faster convergence than FAT.

The completed proof of Theorem 4.3 is given in Appendix B.3 and the empirical verification is presented in Appendix E.7.

The following section will comprehensively confirm that SFAT can reach to a more robust optimum.

Note that, the theorem we deduce here follows the same assupmitions #OTHEREFR and is mainly to compare the difference with the early convergence analysis in federated learning.

Combating Exacerbated Heterogeneity For Robust Models In Federated Learning

Abstract: Privacy and security concerns in real-world applications have led to the development of adversarially robust federated models. However, the straightforward combination between adversarial training and federated learning in one framework can lead to the undesired robustness deterioration. We discover that the attribution behind this phenomenon is that the generated adversarial data could exacerbate the data heterogeneity among local clients, making the wrapped federated learning perform poorly. To deal with this problem, we propose a novel framework called Slack Federated Adversarial Training (SFAT), assigning the client-wise slack during aggregation to combat the intensified heterogeneity. Theoretically, we analyze the convergence of the proposed method to properly relax the objective when combining federated learning and adversarial training. Experimentally, we verify the rationality and effectiveness of SFAT on various benchmarked and real-world datasets with different adversarial training and federated optimization methods.

Sgd And Hogwild! Convergence Without The Bounded Gradients Assumption

Abstract: Stochastic gradient descent (SGD) is the optimization algorithm of choice in many machine learning applications such as regularized empirical risk minimization and training deep neural networks. The classical convergence analysis of SGD is carried out under the assumption that the norm of the stochastic gradient is uniformly bounded. While this might hold for some loss functions, it is always violated for cases where the objective function is strongly convex. In (Bottou et al.,2016), a new analysis of convergence of SGD is performed under the assumption that stochastic gradients are bounded with respect to the true gradient norm. Here we show that for stochastic problems arising in machine learning such bound always holds; and we also propose an alternative convergence analysis of SGD with diminishing learning rate regime, which results in more relaxed conditions than those in (Bottou et al.,2016). We then move on the asynchronous parallel setting, and prove convergence of Hogwild! algorithm in the same regime, obtaining the first convergence results for this method in the case of diminished learning rate.


In fully-supervised setting, the full training set of each task in Super-GLUE #REFR is employed during the prompt tuning process.

SuperGLUE benchmark #OTHEREFR contains 8 challenging natural language understanding (NLU) tasks.

We also include the few-shot version of SuperGLUE, FewGLUE benchmark #OTHEREFR to test the ability of our algorithm, which consists of 32 training samples in each dataset on SuperGLUE.

Following #OTHEREFR Liu et al., 2021b) , we report results on 7 of 8 NLU tasks in few-shot settings.

Besides, in the model selection part, we adopt the whole validation set.

As few-shot learning ability of prompt tuning can reduce the cost of annotations in real-world applications, following #OTHEREFR Liu et al., 2021b) , we also test our algorithm under fewshot settings.

To be specific, we use the training set provided by FewGLUE #OTHEREFR , the few-shot version of SuperGLUE, containing 32 training pairs in each task.

Besides, we use the same version of the development set as (Liu et al., 2021b) to select models, which are created by randomly choosing 32 unused training pairs.

Ptp: Boosting Stability And Performance Of Prompt Tuning With Perturbation-Based Regularizer

Abstract: Recent studies show that prompt tuning can better leverage the power of large language models than fine-tuning on downstream natural language understanding tasks. However, the existing prompt tuning methods have training instability issues, as the variance of scores under different random seeds is quite large. To address this critical problem, we first investigate and find that the loss landscape of vanilla prompt tuning is precipitous when it is visualized, where a slight change of input data can cause a big fluctuation in the loss landscape. This is an essential factor that leads to the instability of prompt tuning. Based on this observation, we introduce perturbation-based regularizers, which can smooth the loss landscape, into prompt tuning. We propose a new algorithm, called Prompt Tuning with Perturbationbased regularizer (PTP), which can not only alleviate training instability dramatically but also boost the performance of prompt tuning. We design two kinds of perturbation-based regularizers, including random-noise-based and adversarial-based. In particular, our proposed perturbations are flexible on both text space and embedding space. Extensive experiments show the effectiveness of our proposed methods in stabilizing the training. Our new algorithms improve the state-of-the-art prompt tuning methods by 1.94% and 2.34% on Su-perGLUE and FewGLUE benchmarks, respectively.

Superglue: A Stickier Benchmark For General-Purpose Language Understanding Systems

Abstract: In the last year, new models and methods for pretraining and transfer learning have driven striking performance improvements across a range of language understanding tasks. The GLUE benchmark, introduced a little over one year ago, offers a single-number metric that summarizes progress on a diverse set of such tasks, but performance on the benchmark has recently surpassed the level of non-expert humans, suggesting limited headroom for further research. In this paper we present SuperGLUE, a new benchmark styled after GLUE with a new set of more difficult language understanding tasks, a software toolkit, and a public leaderboard. SuperGLUE is available at https://super.gluebenchmark.com.

prompt tuning process
fully-supervised setting


As shown in equation #REFR , the index function is convergent as the radius R tends to infinity, which explains the new configuration can still achieve good reconstructions. Based on the results shown in Fig.

In this example, we use the network trained by the MNIST dataset with N i = 16 to reconstruct Latin letters "D", "S" and "M" with high noise levels 50% and 100%.

In addition, different from the configuration for the training data, we use 60 receivers equally distributed on a circular curve with a radius 5 centered at the origin to collect the data for the scattered field.

12 , it can be seen that the proposed DSM-DL is very robust to high-level noise as the noise is smoothed in the DSM process.

We can also observe that the reconstruction of the thinner part of the letters is more likely to be distorted, which is mainly because the thinner part contributes less to the scattered field.

A Direct Sampling-Based Deep Learning Approach For Inverse Medium Scattering Problems

Abstract: In this work, we focus on the inverse medium scattering problem (IMSP), which aims to recover unknown scatterers based on measured scattered data. Motivated by the efficient direct sampling method (DSM) introduced in [23], we propose a novel direct sampling-based deep learning approach (DSM-DL) for reconstructing inhomogeneous scatterers. In particular, we use the U-Net neural network to learn the relation between the index functions and the true contrasts. Our proposed DSM-DL is computationally efficient, robust to noise, easy to implement, and able to naturally incorporate multiple measured data to achieve high-quality reconstructions. Some representative tests are carried out with varying numbers of incident waves and different noise levels to evaluate the performance of the proposed method. The results demonstrate the promising benefits of combining deep learning techniques with the DSM for IMSP.

Direct Sampling Method For Diffusive Optical Tomography

Abstract: In this work, we are concerned with the diffusive optical tomography (DOT) problem in the case when only one or two pairs of Cauchy data are available. We propose a simple and efficient direct sampling method to locate inhomogeneities inside a homogeneous background and solve the DOT problem in both full and limited aperture cases. This new method is easy to implement and less expensive computationally. Numerical experiments demonstrate its effectiveness and robustness against noise in the data. This provides a new promising numerical strategy for the DOT problem.


Recent research from molecular biology, to multi-robot path planning #REFR have demonstrated promising performance of GNNs to capture the interaction between a node in the graph for node and edge classification and message passing.

This property yields the rising research in capturing tactile sensor data using GNN #OTHEREFR , #OTHEREFR for in-hand manipulation.

The properties of GNNs in capturing inter-node reactions inspired researchers to use GNN as a control algorithm.

The strength of GNNs has been recently utilized in tactile sensing. Garcia-Garcia et al.

#OTHEREFR proposed a GCN-based approach to capture taxel reading from BioTac SP taxel to binary classify grasps as stable or slippery ones. Funabashi et al.

#OTHEREFR proposed a GCN-based pipeline to extract geodesical features from the tactile data from distributed taxels.

Tacgnn: Learning Tactile-Based In-Hand Manipulation With A Blind Robot Using Hierarchical Graph Neural Network

Abstract: In this paper, we propose a novel framework for tactile-based dexterous manipulation learning with a blind anthropomorphic robotic hand, i.e. without visual sensing. First, object-related states were extracted from the raw tactile signals by a graph-based perception model-TacGNN. The resulting tactile features were then utilized in the policy learning of an in-hand manipulation task in the second stage. This method was examined by a Baoding ball task-simultaneously manipulating two spheres around each other by 180 degrees in hand. We conducted experiments on object states prediction and in-hand manipulation using a reinforcement learning algorithm (PPO). Results show that TacGNN is effective in predicting objectrelated states during manipulation by decreasing the RMSE of prediction to 0.096cm comparing to other methods, such as MLP, CNN, and GCN. Finally, the robot hand could finish an in-hand manipulation task solely relying on the robotic own perception-tactile sensing and proprioception. In addition, our methods are tested on three tasks with different difficulty levels and transferred to the real robot without further training. https://sites.google.com/view/tacgnn

A Comprehensive Survey On Graph Neural Networks

Abstract: Deep learning has revolutionized many machine learning tasks in recent years, ranging from image classification and video processing to speech recognition and natural language understanding. The data in these tasks are typically represented in the Euclidean space. However, there is an increasing number of applications, where data are generated from non-Euclidean domains and are represented as graphs with complex relationships and interdependency between objects. The complexity of graph data has imposed significant challenges on the existing machine learning algorithms. Recently, many studies on extending deep learning approaches for graph data have emerged. In this article, we provide a comprehensive overview of graph neural networks (GNNs) in data mining and machine learning fields. We propose a new taxonomy to divide the state-of-the-art GNNs into four categories, namely, recurrent GNNs, convolutional GNNs, graph autoencoders, and spatial-temporal GNNs. We further discuss the applications of GNNs across various domains and summarize the open-source codes, benchmark data sets, and model evaluation of GNNs. Finally, we propose potential research directions in this rapidly growing field.


Comparison of different backbones and alignment methods on LVIS v1.0 #REFR with close-set settings.

We verify the effectiveness of grid-level and image-level alignment for undersampled categories for both close-set detection (Table 4) and open-set detection ( Table 5 ). We follow the same settings in Sec.

4.2 except that multi-scaling training and random cropping augmentation are not used here.

Among the experiments, "wo align" denotes using the original design of FCOS only with different backbones Table 4 .

The top section compares supervised (ImageNet #OTHEREFR ) and unsupervised (SwAV #OTHEREFR ) pretrained visual models with CLIP as unsupervised visual-language pretrained model.

The bottom section compares different alignment methods based on the ResNet50 version of pretrained CLIP image encoder.

"GridCLIP-R50 * " uses the CLIP ResNet50 without the MHSA layer, while "GridCLIP-R50" uses the whole image encoder of CLIP ResNet50. Figure 3 .

The AP on LVIS v1.0 over categories sorted by frequency in ascending order.

(a) uses on the close-set setting, only containing 461 "common" categories and 405 "frequent" categories.

Gridclip: One-Stage Object Detection By Grid-Level Clip Representation Learning

Abstract: A vision-language foundation model pretrained on very large-scale image-text paired data has the potential to provide generalizable knowledge representation for downstream visual recognition and detection tasks, especially on supplementing the undersampled categories in downstream model training. Recent studies utilizing CLIP for object detection have shown that a two-stage detector design typically outperforms a one-stage detector, while requiring more expensive training resources and longer inference time. In this work, we propose a one-stage detector GridCLIP that narrows its performance gap to those of twostage detectors, with approximately 43× and 5× faster than its two-stage counterpart (ViLD) in the training and test process respectively. GridCLIP learns grid-level representations to adapt to the intrinsic principle of one-stage detection learning by expanding the conventional CLIP imagetext holistic mapping to a more fine-grained, grid-text alignment. This differs from the region-text mapping in two-stage detectors that apply CLIP directly by treating regions as images. Specifically, GridCLIP performs Grid-level Alignment to adapt the CLIP image-level representations to gridlevel representations by aligning to CLIP category representations to learn the annotated (especially frequent) categories. To learn generalizable visual representations of broader categories, especially undersampled ones, we perform Image-level Alignment during training to propagate broad pre-learned categories in the CLIP image encoder from the image-level to the grid-level representations. Experiments show that the learned CLIP-based grid-level representations boost the performance of undersampled (infrequent and novel) categories, reaching comparable detection performance on the LVIS benchmark.

Lvis: A Dataset For Large Vocabulary Instance Segmentation

Abstract: Progress on object detection is enabled by datasets that focus the research community’s attention on open challenges. This process led us from simple images to complex scenes and from bounding boxes to segmentation masks. In this work, we introduce LVIS (pronounced ‘el-vis’): a new dataset for Large Vocabulary Instance Segmentation. We plan to collect 2.2 million high-quality instance segmentation masks for over 1000 entry-level object categories in 164k images. Due to the Zipfian distribution of categories in natural images, LVIS naturally has a long tail of categories with few training samples. Given that state-of-the-art deep learning methods for object detection perform poorly in the low-sample regime, we believe that our dataset poses an important and exciting new scientific challenge. LVIS is available at http://www.lvisdataset.org.


However, such classical approaches usually start without any initialization while SMART effectively utilizes shape-dependent initialization learned by neural networks (BSP-Net) #REFR .

#OTHEREFR 's work is based on visibility (line of sight), which is one of the volumetric properties. Shapira et al.

#OTHEREFR define a shape diameter function that expresses the diameter of the object's volume in the nearby point on the surface for consistent mesh partitioning. Liu et al.

#OTHEREFR design a part-aware surface metric that considers the volumetric context when encoding part information. Kaick et al.

#OTHEREFR merge weakly convex components taking account of the volumetric profile of the parts to obtain the final segmentation.

In the sense of exploiting volumetric information to capture part-level segmentation, SMART is similar to these approaches.

Moreover, we propose an additional step to refine the errors and failures of these approaches. Efficient Search for 3D Applications.

Efficient search is crucial for tackling high-dimensional search spaces since we cannot exhaustively search all the possible solutions.

Continuity and extra dimensionality in 3D data make the search space extremely large, requiring dedicated algorithms for efficient searches.

For this purpose beam search #OTHEREFR , monte carlo tree search (MCTS) #OTHEREFR 36] and learning #OTHEREFR approaches have been explored to solve various 3D computer vision tasks such as shape reconstruction, scene detection and convex decomposition. Sharma et al.

#OTHEREFR propose a neural shape parser that learns to generate CSG grammar to challenge the huge search spaces.

Split, Merge, And Refine: Fitting Tight Bounding Boxes Via Learned Over-Segmentation And Iterative Search

Abstract: Figure 1: Our framework dubbed SMART finds a set of tight bounding boxes of a 3D shape. Given a 3D shape represented as a volumetric mesh, it first takes over-segmentation from the output of BSP-Net [8] and post-processing and then performs hierarchical merging with tightness-aware criteria (second row). Then, it refines the bounding box parameters with a sequence of actions in a discrete space (third row). The results are further improved with long-sighted exploration by our MCTS accelerated with our technique (last row).

Bsp-Net: Generating Compact Meshes Via Binary Space Partitioning

Abstract: Polygonal meshes are ubiquitous in the digital 3D domain, yet they have only played a minor role in the deep learning revolution. Leading methods for learning generative models of shapes rely on implicit functions, and generate meshes only after expensive iso-surfacing routines. To overcome these challenges, we are inspired by a classical spatial data structure from computer graphics, Binary Space Partitioning (BSP), to facilitate 3D learning. The core ingredient of BSP is an operation for recursive subdivision of space to obtain convex sets. By exploiting this property, we devise BSP-Net, a network that learns to represent a 3D shape via convex decomposition. Importantly, BSP-Net is unsupervised since no convex shape decompositions are needed for training. The network is trained to reconstruct a shape using a set of convexes obtained from a BSP-tree built on a set of planes. The convexes inferred by BSP-Net can be easily extracted to form a polygon mesh, without any need for iso-surfacing. The generated meshes are compact (i.e., low-poly) and well suited to represent sharp geometry; they are guaranteed to be watertight and can be easily parameterized. We also show that the reconstruction quality by BSP-Net is competitive with state-of-the-art methods while using much fewer primitives. Code is available at https://github.com/czq142857/BSP-NET-original.


Moreover, we observe that HouseGAN #REFR cannot produce very discriminative rooms, leading to similar generation results for different types of rooms.

Next, we reshape it to a tensor of size 8×32×32.

Thus, we use a shared three-layer CNN to convert it to a feature of size 16×32×32, followed by two rounds of Conv-MPN and downsampling.

Lastly, we use another three-layer CNN to convert each room feature into a 128-d vector − → d r .

To classify ground-truth samples from the generated ones, we sum-pool over all the room vectors and then apply a single linear layer to produce a scalard 1 , which can be expressed as follows,

To provide a more diverse generation for different rooms, we propose a novel node classification loss to learn more discriminative classspecific node representations.

Specifically, we sum pool over all the room vectors and add another single linear layer to output a 10-d one-hot vectord 2 , classifying generated rooms to the corresponding room labels,

We use the binary cross-entropy loss between the real room label d 2 and the predicted labeld 2 .

Graph Transformer Gans For Graph-Constrained House Generation

Abstract: We present a novel graph Transformer generative adversarial network (GTGAN) to learn effective graph node relations in an end-to-end fashion for the challenging graphconstrained house generation task. The proposed graph-Transformer-based generator includes a novel graph Transformer encoder that combines graph convolutions and selfattentions in a Transformer to model both local and global interactions across connected and non-connected graph nodes. Specifically, the proposed connected node attention (CNA) and non-connected node attention (NNA) aim to capture the global relations across connected nodes and non-connected nodes in the input graph, respectively. The proposed graph modeling block (GMB) aims to exploit local vertex interactions based on a house layout topology. Moreover, we propose a new node classification-based discriminator to preserve the high-level semantic and discriminative node features for different house components. Finally, we propose a novel graph-based cycle-consistency loss that aims at maintaining the relative spatial relationships between ground truth and predicted graphs. Experiments on two challenging graph-constrained house generation tasks (i.e., house layout and roof generation) with two public datasets demonstrate the effectiveness of GTGAN in terms of objective quantitative scores and subjective visual realism. New state-of-the-art results are established by large margins on both tasks.

House-Gan: Relational Generative Adversarial Networks For Graph-Constrained House Layout Generation

Abstract: This paper proposes a novel graph-constrained generative adversarial network, whose generator and discriminator are built upon relational architecture. The main idea is to encode the constraint into the graph structure of its relational networks. We have demonstrated the proposed architecture for a new house layout generation problem, whose task is to take an architectural constraint as a graph (i.e., the number and types of rooms with their spatial adjacency) and produce a set of axis-aligned bounding boxes of rooms. We measure the quality of generated house layouts with the three metrics: the realism, the diversity, and the compatibility with the input graph constraint. Our qualitative and quantitative evaluations over 117,000 real floorplan images demonstrate that the proposed approach outperforms existing methods and baselines. We will publicly share all our code and data.


SMNA #REFR uses visual-textual co-grounding module that encodes the past instructions and the instructions and actions to be done.

Seq2Seq #OTHEREFR uses sequence-to-sequence action prediciton to generate actions from the agent trajectory.

Speaker-Follower #OTHEREFR uses the speaker model to augment natural language instructions and evaluate the candidate action sequence.

FAST #OTHEREFR uses both local and global signals to look forward the unobserved environment during exploration and backtrack to the originally visited nodes when needed.

SMNA also uses a progress monitor to estimate the current progress of the agent relative to the total instructions. Regretful-Agent #OTHEREFR improves SMNA via two modules.

The regret module decides whether to continue to explore or rollback to previous state by a learned policy, and the progress marker decides the direction the agent should head to by selecting visited nodes with progress estimates.

RCM #OTHEREFR applies reinforcement learning to enforce the global matching between the agent trajectory and the given natural language instruction.

Via cycle-reconstruction reward, RCM allows the agent to comprehend the natural language instruction and penalize paths that do not match with the given instructions.

FAST-MATTN #OTHEREFR introduces a Navigator-Pointer model to both navigate to the target point and to localize the object from the navigation point according to the language guidance.

Meta-Explore: Exploratory Hierarchical Vision-And-Language Navigation Using Scene Object Spectrum Grounding

Abstract: The main challenge in vision-and-language navigation (VLN) is how to understand natural-language instructions in an unseen environment. The main limitation of conventional VLN algorithms is that if an action is mistaken, the agent fails to follow the instructions or explores unnecessary regions, leading the agent to an irrecoverable path. To tackle this problem, we propose Meta-Explore, a hierarchical navigation method deploying an exploitation policy to correct misled recent actions. We show that an exploitation policy, which moves the agent toward a well-chosen local goal among unvisited but observable states, outperforms a method which moves the agent to a previously visited state. We also highlight the demand for imagining regretful explorations with semantically meaningful clues. The key to our approach is understanding the object placements around the agent in spectral-domain. Specifically, we present a novel visual representation, called scene object spectrum (SOS), which performs category-wise 2D Fourier transform of detected objects. Combining exploitation policy and SOS features, the agent can correct its path by choosing a promising local goal. We evaluate our method in three VLN benchmarks: R2R, SOON, and REVERIE. Meta-Explore outperforms other baselines and shows significant generalization performance. In addition, local goal search using the proposed spectral-domain SOS features significantly improves the success rate by 17.

Self-Monitoring Navigation Agent Via Auxiliary Progress Estimation

Abstract: The Vision-and-Language Navigation (VLN) task entails an agent following navigational instruction in photo-realistic unknown environments. This challenging task demands that the agent be aware of which instruction was completed, which instruction is needed next, which way to go, and its navigation progress towards the goal. In this paper, we introduce a self-monitoring agent with two complementary components: (1) visual-textual co-grounding module to locate the instruction completed in the past, the instruction required for the next action, and the next moving direction from surrounding images and (2) progress monitor to ensure the grounded instruction correctly reflects the navigation progress. We test our self-monitoring agent on a standard benchmark and analyze our proposed approach through a series of ablation studies that elucidate the contributions of the primary components. Using our proposed method, we set the new state of the art by a significant margin (8% absolute increase in success rate on the unseen test set). Code is available at this https URL .


We present the axiomatic framework introduced by Biswas and Enea #REFR for defining isolation levels.

Isolation levels are defined as logical constraints, called axioms, over histories, which are an abstract representation of the interaction between a program and the database in an execution.

Programs interact with a database by issuing transactions formed of begin, commit, abort, read and write instructions.

The effect of executing one such instruction is represented using an event , type where is an identifier and type is a type.

There are five types of events: begin, commit, abort, read( ) for reading the global variable , and write( , ) for writing value to . E denotes the set of events.

Dynamic Partial Order Reduction For Checking Correctness Against Transaction Isolation Levels

Abstract: Modern applications, such as social networking systems and e-commerce platforms are centered around using large-scale databases for storing and retrieving data. Accesses to the database are typically enclosed in transactions that allow computations on shared data to be isolated from other concurrent computations and resilient to failures. Modern databases trade isolation for performance. The weaker the isolation level is, the more behaviors a database is allowed to exhibit and it is up to the developer to ensure that their application can tolerate those behaviors. In this work, we propose stateless model checking algorithms for studying correctness of such applications that rely on dynamic partial order reduction. These algorithms work for a number of widely-used weak isolation levels, including Read Committed, Causal Consistency, Snapshot Isolation, and Serializability. We show that they are complete, sound and optimal, and run with polynomial memory consumption in all cases. We report on an implementation of these algorithms in the context of Java Pathfinder applied to a number of challenging applications drawn from the literature of distributed systems and databases.

On The Complexity Of Checking Transactional Consistency

Abstract: Transactions simplify concurrent programming by enabling computations on shared data that are isolated from other concurrent computations and are resilient to failures. Modern databases provide different consistency models for transactions corresponding to different tradeoffs between consistency and availability. In this work, we investigate the problem of checking whether a given execution of a transactional database adheres to some consistency model. We show that consistency models like read committed, read atomic, and causal consistency are polynomial-time checkable while prefix consistency and snapshot isolation are NP-complete in general. These results complement a previous NP-completeness result concerning serializability. Moreover, in the context of NP-complete consistency models, we devise algorithms that are polynomial time assuming that certain parameters in the input executions, e.g., the number of sessions, are fixed. We evaluate the scalability of these algorithms in the context of several production databases.


B. Proof Of Concept: A Conditional Tabular Gan

Distributing a tabular GAN discGAN models are written with the TensorFlow library #REFR , which provides the tf.distrbute [15] API to distribute model training across GPUs and machines.

These results are encouraging as the generated data both quantitatively and qualitatively resembles real data; however, it is noteworthy that the standard deviation for all unit types and ethnicities is smaller in the generated data than real data, significantly so for the Native American class, which had only 12 real examples in our training data.

A 2D discGAN uses a default noise input of 50 followed by 2 64-node dense layers with LeakyReLU activation and a dropout layer (dropout rate of 0.1).

Batch sizes of 16, 32, and 64 were used, and 32 produced the most accurate results.

The following figures contain the output of the 2D GAN:

To demonstrate a tabular GAN in a distributed environment, we provisioned two GPU's and utilized tf.distribute's MultiWorkerMirroredStrategy which enables synchronous and distributed training using multiple GPUs on a single machine.

MultiWorkerMirroredStrategy can distribute training across multiple workers, each with "potentially multiple GPUs." [15] More specifically, the strategy replicates and updates all model parameters, weights, and computations to each device (hence it being a "mirrored" strategy), trains on each device, then uses distributed collective implementation -namely all-reduce.

All-reduce is a fused algorithm that TensorFlow uses to "aggregate tensors across all the devices by adding them up, and makes them available on each device." [15] TensorFlow uses the NVIDIA Collec- The strategy distributes discGAN by allowing the GPU workers to train together.

Specifically, the devices divide the training data at each step before running forward and backward propagation separately on a mirrored copy of the the GAN models and their parameters on each device.

The strategy then uses all-reduce to aggregate the distributed gradients and sends updated variable weights and a single aggregated gradient to all devices before starting the next training step.

Distributed Conditional Gan (Discgan) For Synthetic Healthcare Data Generation

Abstract: In this paper, we propose a distributed Generative Adversarial Networks (discGANs) to generate synthetic tabular data specific to the healthcare domain. While using GANs to generate images has been well studied, little to no attention has been given to generation of tabular data. Modeling distributions of discrete and continuous tabular data is a non-trivial task with high utility. We applied discGAN to model non-Gaussian multi-modal healthcare data. We generated 249,000 synthetic records from original 2027 eICU dataset and we evaluated the performance of the model using machine learning efficacy, the Kolmogorov-Smirnov test (KS test) for continuous variables, and chi-squared test for discrete variables. Our results show that discGAN was able to generate data with distributions similar to the real data.

Tensorflow: Large-Scale Machine Learning On Heterogeneous Distributed Systems

Abstract: TensorFlow is an interface for expressing machine learning algorithms, and an implementation for executing such algorithms. A computation expressed using TensorFlow can be executed with little or no change on a wide variety of heterogeneous systems, ranging from mobile devices such as phones and tablets up to large-scale distributed systems of hundreds of machines and thousands of computational devices such as GPU cards. The system is flexible and can be used to express a wide variety of algorithms, including training and inference algorithms for deep neural network models, and it has been used for conducting research and for deploying machine learning systems into production across more than a dozen areas of computer science and other fields, including speech recognition, computer vision, robotics, information retrieval, natural language processing, geographic information extraction, and computational drug discovery. This paper describes the TensorFlow interface and an implementation of that interface that we have built at Google. The TensorFlow API and a reference implementation were released as an open-source package under the Apache 2.0 license in November, 2015 and are available at www.tensorflow.org.


We state an important theorem without proof, details of which can be found in #REFR .

X 1 := 1 ε 2 σ(εw x)σ(εw y), then X 1 ψ 1 ≤ C ψ,d , • and if X 2 := a 2 σ (1) (εw x)σ (1) (εw y) x, y , then X 2 ψ 1 ≤ C ψ,1 . Proof.

Since |X 1 | ≤ |w x| |w y| = w 2 2 , then by definition, X 1 ψ 1 = inf s > 0 | E X 1 exp |X 1 | s ≤ 2 = inf s > 0 | E w exp |σ(w x)σ(w y)| s ≤ 2 ≤ inf s > 0 | E w exp w 2 2 s ≤ 2 , hence X 1 ψ 1 ≤ C ψ,d . (A.3)

|X 2 | ≤ a 2 , hence X 2 ψ 1 ≤ C ψ,1 . (A.4)

Theorem 3 (Bernstein's inequality). Let {X k } m k=1 be i.i.d.

sub-exponential random variables satisfying EX 1 = µ, then for any η ≥ 0, we have P 1 m m k=1 X k − µ ≥ η ≤ 2 exp −C 0 m min η 2 X 1 2 ψ 1 , η X 1 ψ 1 ,

Proposition 1 (Upper and lower bounds of initial parameters). Given any δ ∈ (0, 1), if

Phase Diagram Of Initial Condensation For Two-Layer Neural Networks

Abstract: The phenomenon of distinct behaviors exhibited by neural networks under varying scales of initialization remains an enigma in deep learning research. In this paper, based on the earlier work by Luo et al. [16], we present a phase diagram of initial condensation for twolayer neural networks. Condensation is a phenomenon wherein the weight vectors of neural networks concentrate on isolated orientations

Introduction To The Non-Asymptotic Analysis Of Random Matrices

Abstract: This is a tutorial on some basic non-asymptotic methods and concepts in random matrix theory. The reader will learn several tools for the analysis of the extreme singular values of random matrices with independent rows or columns. Many of these methods sprung off from the development of geometric functional analysis since the 1970's. They have applications in several fields, most notably in theoretical computer science, statistics and signal processing. A few basic applications are covered in this text, particularly for the problem of estimating covariance matrices in statistics and for validating probabilistic constructions of measurement matrices in compressed sensing. These notes are written particularly for graduate students and beginning researchers in different areas, including functional analysts, probabilists, theoretical statisticians, electrical engineers, and theoretical computer scientists.


They found good results balancing between overall accuracy and fairness and performed better than other fairness methods #REFR .

#OTHEREFR proposed a debiasing approach when creating embeddings by incorporating the loss function in the training process with a pre-trained binary (e.g., positive or negative) classifier.

Boulitsakis-Logothetis #OTHEREFR proposed a modification to the Naive Bayes algorithm that gives guarantees on the statistical parity across protected attributes.

This paper found benefits in terms of increases in fairness measures across protected classes but found that accuracy would often be degraded. Bechavod et al.

#OTHEREFR proposed a method of penalizing unfairness which was inspired by regularization.

In this way, they have introduced a data-dependent penalty to the learning process.

As compared to previous work, we looked at the impact of noisy training data on the level of fairness achieved by models trained with such data, compared the robustness to noise of different models, and also looked at a transfer learning scenario.

On The Impact Of Data Quality On Image Classification Fairness

Abstract: With the proliferation of algorithmic decision-making, increased scrutiny has been placed on these systems. This paper explores the relationship between the quality of the training data and the overall fairness of the models trained with such data in the context of supervised classification. We measure key fairness metrics across a range of algorithms over multiple image classification datasets that have a varying level of noise in both the labels and the training data itself. We describe noise in the labels as inaccuracies in the labelling of the data in the training set and noise in the data as distortions in the data, also in the training set. By adding noise to the original datasets, we can explore the relationship between the quality of the training data and the fairness of the output of the models trained on that data.

Fairness Beyond Disparate Treatment&Disparate Impact: Learning Classification Without Disparate Mistreatment

Abstract: Automated data-driven decision making systems are increasingly being used to assist, or even replace humans in many settings. These systems function by learning from historical decisions, often taken by humans. In order to maximize the utility of these systems (or, classifiers), their training involves minimizing the errors (or, misclassifications) over the given historical data. However, it is quite possible that the optimally trained classifier makes decisions for people belonging to different social groups with different misclassification rates (e.g., misclassification rates for females are higher than for males), thereby placing these groups at an unfair disadvantage. To account for and avoid such unfairness, in this paper, we introduce a new notion of unfairness, disparate mistreatment, which is defined in terms of misclassification rates. We then propose intuitive measures of disparate mistreatment for decision boundary-based classifiers, which can be easily incorporated into their formulation as convex-concave constraints. Experiments on synthetic as well as real world datasets show that our methodology is effective at avoiding disparate mistreatment, often at a small cost in terms of accuracy.


We also compare our findings with the previously known results of Alekasandrov et al #REFR .

Our report is organized as follows: In Section 1.1, we provide the motivation behind our report.

In Section 2, we provide the necessary notations and preliminaries used in our proofs.

Section 3 briefly mentions the results presented in #OTHEREFR , while also defining the notions of strategyproofness and envy-freeness.

Next, in Section 4, we present our weighted LIKE and weighted Balanced LIKE mechanisms, along with a table summarizing our results.

We introduce the concept of the game tree, which helps in calculating expected utility in our weighted Balanced LIKE mechanism, in Section 5.

In Section 6, we present our results related to the strategy-proofness of the weighted LIKE and weighted Balanced LIKE mechanisms.

Finally, in Section 7, we present our results related to the envy-freeness of the weighted LIKE and weighted Balanced LIKE mechanisms.

Online Fair Division With Arbitrary Entitlements

Abstract: The division of goods in the online realm poses opportunities and challenges. While innovative mechanisms can be developed, uncertainty about the future may hinder effective solutions. This project aims to explore fair distribution models for goods among agents with arbitrary entitlements, specifically addressing food charity challenges in the real world. Building upon prior work in [AAGW15], which focuses on equal entitlements, our project seeks to better understand the proofs of the theorems mentioned in that paper, which currently only provide proof sketches. Our approach employs different proof techniques from those presented in [AAGW15].

Online Fair Division: Analysing A Food Bank Problem

Abstract: We study an online model of fair division designed to capture features of a real world charity problem. We consider two simple mechanisms for this model in which agents simply declare what items they like. We analyse axiomatic properties of these mechanisms such as strategy-proofness and envy freeness. Finally, we perform a competitive analysis and compute the price of anarchy.


We evaluated our reconstructed images using PSNR and SSIM #REFR as metrics, where both metrics are of higher values indicating better quality of the recovered images.

We used the most widely used DIV2K #OTHEREFR for training, which has 900 images, of which 800 are used as the training set, and 100 are the validation set.

LR images were aquired by double triple down-sampling We also tested our model on five common benchmark datasets: Set5 #OTHEREFR , Set14 #OTHEREFR , BSD100 #OTHEREFR , Urban100 #OTHEREFR , and Manga109 #OTHEREFR .

A High-Frequency Focused Network For Lightweight Single Image Super-Resolution

Abstract: Lightweight neural networks for single-image superresolution (SISR) tasks have made substantial breakthroughs in recent years. Compared to low-frequency information, highfrequency detail is much more difficult to reconstruct. Most SISR models allocate equal computational resources for lowfrequency and high-frequency information, which leads to redundant processing of simple low-frequency information and inadequate recovery of more challenging high-frequency information. We propose a novel High-Frequency Focused Network (HFFN) through High-Frequency Focused Blocks (HFFBs) that selectively enhance high-frequency information while minimizing redundant feature computation of low-frequency information. The HFFB effectively allocates more computational resources to the more challenging reconstruction of high-frequency information. Moreover, we propose a Local Feature Fusion Block (LFFB) effectively fuses features from multiple HFFBs in a local region, utilizing complementary information across layers to enhance feature representativeness and reduce artifacts in reconstructed images. We assess the efficacy of our proposed HFFN on five benchmark datasets and show that it significantly enhances the super-resolution performance of the network. Our experimental results demonstrate state-of-the-art performance in reconstructing high-frequency information while using a low number of parameters.

Sketch-Based Manga Retrieval Using Manga109 Dataset

Abstract: Manga (Japanese comics) are popular worldwide. However, current e-manga archives offer very limited search support, i.e., keyword-based search by title or author. To make the manga search experience more intuitive, efficient, and enjoyable, we propose a manga-specific image retrieval system. The proposed system consists of efficient margin labeling, edge orientation histogram feature description with screen tone removal, and approximate nearest-neighbor search using product quantization. For querying, the system provides a sketch-based interface. Based on the interface, two interactive reranking schemes are presented: relevance feedback and query retouch. For evaluation, we built a novel dataset of manga images, Manga109, which consists of 109 comic books of 21,142 pages drawn by professional manga artists. To the best of our knowledge, Manga109 is currently the biggest dataset of manga images available for research. Experimental results showed that the proposed framework is efficient and scalable (70 ms from 21,142 pages using a single computer with 204 MB RAM).


The Soft Layer Ordering technique #REFR identifies the limitations of traditional fixed-order sharing approaches in MTL models and proposes learning task-specific scaling parameters to enable a flexible ordering of shared layers for each task.

Differently, our work introduces two kinds of complementary fusion modules to separately model task specific learning and shared learning.

Also, In addition to explicitly introducing shared modules for learning commonalities across all tasks, we leverage direct task-pair fusion, based on input, to maximize the flexibility of knowledge sharing.

Neural Architecture Search (NAS) #OTHEREFR methods have been applied to Multi-Task Learning (MTL) to automatically learn the structure of the models.

Branched Multi-Task Networks #OTHEREFR generate a tree structure by clustering tasks based on affinity scores and assigning dissimilar tasks to the different branches.

#OTHEREFR utilizes Gumbel-Softmax sampling for the branching operation instead of pre-calculated affinity scores, making the network endto-end trainable.

AdaShare #OTHEREFR learns a task-specific policy to select which layers to execute for every particular task.

Sub-Network Routing (SNR) #OTHEREFR splits shared layers into sub-networks and learns their connections with latent variables.

NAS methods eliminate a significant amount of manual work and improve the flexibility of the sharing patterns in the MTL model.

However, as an exhaustive search of all possible model configurations would be combinatorially complex, these methods often rely on quite simplified assumptions such as branching #OTHEREFR , routing #OTHEREFR , layer ordering #OTHEREFR , layer selecting #OTHEREFR etc. to limit the search space. Additionally, the generated structures don't adjust to individual examples.

In addition to works focusing on MTL architecture design, another line of works aims to improve MTL optimization.

Adatt: Adaptive Task-To-Task Fusion Network For Multitask Learning In Recommendations

Abstract: Multi-task learning (MTL) aims at enhancing the performance and efficiency of machine learning models by training them on multiple tasks simultaneously. However, MTL research faces two challenges: 1) modeling the relationships between tasks to effectively share knowledge between them, and 2) jointly learning task-specific and shared knowledge. In this paper, we present a novel model Adaptive Task-to-Task Fusion Network (AdaTT) to address both challenges. AdaTT is a deep fusion network built with task specific and optional shared fusion units at multiple levels. By leveraging a residual mechanism and gating mechanism for task-to-task fusion, these units adaptively learn shared knowledge and task specific knowledge. To evaluate the performance of AdaTT, we conduct experiments on a public benchmark and an industrial recommendation dataset using various task groups. Results demonstrate AdaTT can significantly outperform existing state-of-the-art baselines. CCS CONCEPTS • Computing methodologies → Multi-task learning; • Information systems → Recommender systems.

Beyond Shared Hierarchies: Deep Multitask Learning Through Soft Layer Ordering

Abstract: Existing deep multitask learning (MTL) approaches align layers shared between tasks in a parallel ordering. Such an organization significantly constricts the types of shared structure that can be learned. The necessity of parallel ordering for deep MTL is first tested by comparing it with permuted ordering of shared layers. The results indicate that a flexible ordering can enable more effective sharing, thus motivating the development of a soft ordering approach, which learns how shared layers are applied in different ways for different tasks. Deep MTL with soft ordering outperforms parallel ordering methods across a series of domains. These results suggest that the power of deep MTL comes from learning highly general building blocks that can be assembled to meet the demands of each task.


Automatic Segmentation With One-Shot Learning

For the other three classes, models trained on the LSUN dataset #REFR are used which contains more than 1 × 10 6 samples for each class.

For each class, we have used two different datasets for evaluation: (i) The first dataset is a large-scale unlabeled dataset on which the GAN model is pre-trained for image synthesis.

And (ii) the second dataset is a small-size labeled dataset wherein the samples have been annotated with segmentation labels and are used to train and evaluate the one-shot learning model.

The first section of Table 1 lists the different unlabeled datasets used for pre-training the StyleGAN models for image synthesis.

The table also mentions the size of the StyleGAN model being pre-trained.

Here, two main datasets are made use of for pre-training the models: For the Face class, we use a StyleGAN2-ADA model that is pre-trained on the Flickr-Faces-HQ dataset #OTHEREFR to generate human faces.

For the Cat class, the Animals Faces-HQ #OTHEREFR dataset is also additionally used.

Self-Supervised One-Shot Learning For For Automatic Segmentation Of Stylegan Images

Abstract: We propose a framework for the automatic one-shot segmentation of synthetic images generated by a StyleGAN. Our framework is based on the observation that the multi-scale hidden features in the GAN generator hold useful semantic information that can be utilized for automatic on-the-fly segmentation of the generated images. Using these features, our framework learns to segment synthetic images using a self-supervised contrastive clustering algorithm that projects the hidden features into a compact space for per-pixel classification. This novel contrastive learner is based on using a pixel-wise swapped prediction loss for image segmentation that leads to faster learning of the feature vectors for one-shot segmentation. We have tested our implementation on a number of standard benchmarks to yield a segmentation performance that not only outperforms the semi-supervised baseline methods by an average wIoU margin of 1.02% but also improves the inference speeds by a factor of 4.5. Finally, we also show the results of using the proposed one-shot learner in implementing BagGAN, a framework for producing annotated synthetic baggage X-ray scans for threat detection. This framework was trained and tested on the PIDRay baggage benchmark to yield a performance comparable to its baseline segmenter based on manual annotations.

Lsun: Construction Of A Large-Scale Image Dataset Using Deep Learning With Humans In The Loop

Abstract: While there has been remarkable progress in the performance of visual recognition algorithms, the state-of-the-art models tend to be exceptionally data-hungry. Large labeled training datasets, expensive and tedious to produce, are required to optimize millions of parameters in deep network models. Lagging behind the growth in model capacity, the available datasets are quickly becoming outdated in terms of size and density. To circumvent this bottleneck, we propose to amplify human effort through a partially automated labeling scheme, leveraging deep learning with humans in the loop. Starting from a large set of candidate images for each category, we iteratively sample a subset, ask people to label them, classify the others with a trained model, split the set into positives, negatives, and unlabeled based on the classification confidence, and then iterate with the unlabeled set. To assess the effectiveness of this cascading procedure and enable further progress in visual recognition research, we construct a new image dataset, LSUN. It contains around one million labeled images for each of 10 scene categories and 20 object categories. We experiment with training popular convolutional networks and find that they achieve substantial performance gains when trained on this dataset.


A very simple form of untargeted adversarial attacks are adversarial noise attacks, which were originally introduced by #REFR .

Noise attacks are applicable to both classification tasks and regression tasks.

They perturb the input data by adding random noise, commonly Gaussian noise or uniform noise.

In the process, the perturbation is normalized and rescaled to the desired size, e.g., with respect to the L ∞ norm.

In addition, the perturbed samples need to be clipped afterwards so that all values are within the valid lower and upper bounds of the input data #OTHEREFR .

Noise attacks require no prior knowledge of the model and thus represent black-box attacks.

Targeted Adversarial Attacks On Wind Power Forecasts

Abstract: In recent years, researchers proposed a variety of deep learning models for wind power forecasting. These models predict the wind power generation of wind farms or entire regions more accurately than traditional machine learning algorithms or physical models. However, latest research has shown that deep learning models can often be manipulated by adversarial attacks. Since wind power forecasts are essential for the stability of modern power systems, it is important to protect them from this threat. In this work, we investigate the vulnerability of two different forecasting models to targeted, semitargeted, and untargeted adversarial attacks. We consider a Long Short-Term Memory (LSTM) network for predicting the power generation of a wind farm and a Convolutional Neural Network (CNN) for forecasting the wind power generation throughout Germany. Moreover, we propose the Total Adversarial Robustness Score (TARS), an evaluation metric for quantifying the robustness of regression models to targeted and semi-targeted adversarial attacks. It assesses the impact of attacks on the model's performance, as well as the extent to which the attacker's goal was achieved, by assigning a score between 0 (very vulnerable) and 1 (very robust). In our experiments, the LSTM forecasting model was fairly robust and achieved a TARS value of over 0.81 for all adversarial attacks investigated. The CNN forecasting model only achieved TARS values below 0.06 when trained ordinarily, and was thus very vulnerable. Yet, its robustness could be significantly improved by adversarial training, which always resulted in a TARS above 0.46.

Foolbox V0.8.0: A Python Toolbox To Benchmark The Robustness Of Machine Learning Models

Abstract: Even todays most advanced machine learning models are easily fooled by almost imperceptible perturbations of their inputs. Foolbox is a new Python package to generate such adversarial perturbations and to quantify and compare the robustness of machine learning models. It is build around the idea that the most comparable robustness measure is the minimum perturbation needed to craft an adversarial example. To this end, Foolbox provides reference implementations of most published adversarial attack methods alongside some new ones, all of which perform internal hyperparameter tuning to find the minimum adversarial perturbation. Additionally, Foolbox interfaces with most popular deep learning frameworks such as PyTorch, Keras, TensorFlow, Theano and MXNet and allows different adversarial criteria such as targeted misclassification and top-k misclassification as well as different distance measures. The code is licensed under the MIT license and is openly available at this https URL . The most up-to-date documentation can be found at this http URL .

untargeted adversarial attacks
adversarial noise attacks


The network is trained using the RAdam optimizer #REFR as it eliminates the need for learning rate warmup during training.

We use the spatial partitioning strategy from #OTHEREFR to partition the adjacency matrix representing the joint connections.

Dilations of 1, 2, and 4 are used in the temporal convolution layers in the GCN-TCN unit in order increase their receptive field.

We use learning rate of 3 × 10 −4 and decay it by a factor of 0.99 every epoch.

The model is regularized using a weight decay of 1e − 5 and a dropout rate of 0.1 in the convolutional layers operating along the temporal dimension.

By performing some test runs we picked the values α = 10 and β = 0.1.

In the SSA unit shown in 2 we use a temporal convolution layer after the spatial attention.

Analogous to this we use a GCN unit to extract the spatial features before extracting temporal attention features.

Spotr: Spatio-Temporal Pose Transformers For Human Motion Prediction

Abstract: 3D human motion prediction is a research area of high significance and a challenge in computer vision. It is useful for the design of many applications including robotics and autonomous driving. Traditionally, autogregressive models have been used to predict human motion. However, these models have high computation needs and error accumulation that make it difficult to use them for realtime applications. In this paper, we present a non-autogressive model for human motion prediction. We focus on learning spatiotemporal representations non-autoregressively for generation of plausible future motions. We propose a novel architecture that leverages the recently proposed Transformers. Human motion involves complex spatio-temporal dynamics with joints affecting the position and rotation of each other even though they are not connected directly. The proposed model extracts these dynamics using both convolutions and the self-attention mechanism. Using specialized spatial and temporal self-attention to augment the features extracted through convolution allows our model to generate spatiotemporally coherent predictions in parallel independent of the activity. Our contributions are threefold: (i) we frame human motion prediction as a sequence-to-sequence problem and propose a non-autoregressive Transformer to forecast a sequence of poses in parallel; (ii) our method is activity agnostic; (iii) we show that despite its simplicity, our approach is able to make accurate predictions, achieving better or comparable results compared to the state-of-theart on two public datasets, with far fewer parameters and much faster inference.

On The Variance Of The Adaptive Learning Rate And Beyond

Abstract: The learning rate warmup heuristic achieves remarkable success in stabilizing training, accelerating convergence and improving generalization for adaptive stochastic optimization algorithms like RMSprop and Adam. Pursuing the theory behind warmup, we identify a problem of the adaptive learning rate -- its variance is problematically large in the early stage, and presume warmup works as a variance reduction technique. We provide both empirical and theoretical evidence to verify our hypothesis. We further propose Rectified Adam (RAdam), a novel variant of Adam, by introducing a term to rectify the variance of the adaptive learning rate. Experimental results on image classification, language modeling, and neural machine translation verify our intuition and demonstrate the efficacy and robustness of RAdam.


In addition, #REFR is an adversarial augmentation method that increases handwritten text images by transforming shapes.

The task of handwritten text generation is similar to that of photographic ancient character generation, aiming to imitate natural handwriting in human style.

The early two-step methods #OTHEREFR generate isolated letters, and then concatenate them to produce a whole word.

These methods rely on handcrafted rules and only generate handwriting with limited variations. Recently, deep generative models directly generate whole-word images.

RNN and GAN were used to generate handwriting in a variety of styles #OTHEREFR .

#OTHEREFR and #OTHEREFR generate text images on the condition of extracting style features in a few-shot setup and textual content of a predefined fixed length.

The above-mentioned methods are designed to generate only Latin characters, and most of them require expensive text annotations to enhance the generation quality.

Agtgan: Unpaired Image Translation For Photographic Ancient Character Generation

Abstract: The study of ancient writings has great value for archaeology and philology. Essential forms of material are photographic characters, but manual photographic character recognition is extremely timeconsuming and expertise-dependent. Automatic classification is therefore greatly desired. However, the current performance is limited due to the lack of annotated data. Data generation is an inexpensive but useful solution to data scarcity. Nevertheless, the diverse glyph shapes and complex background textures of photographic ancient characters make the generation task difficult, leading to unsatisfactory results of existing methods. To this end, we propose an unsupervised generative adversarial network called AGTGAN in this paper. By explicitly modeling global and local glyph shape style, followed by a stroke-aware texture transfer and an associate adversarial learning mechanism, our method can generate characters with diverse glyphs and realistic textures. We evaluate our method on photographic ancient character datasets, e.g., OBC306 and CSDD. Our method outperforms other state-of-the-art methods in terms of various metrics and performs much better in terms of the diversity and authenticity of generated samples. With our generated images, experiments on the largest photographic oracle bone character dataset show that our method can achieve a significant increase in classification accuracy, up to 16.34%. The source code is

Learn To Augment: Joint Data Augmentation And Network Optimization For Text Recognition

Abstract: Handwritten text and scene text suffer from various shapes and distorted patterns. Thus training a robust recognition model requires a large amount of data to cover diversity as much as possible. In contrast to data collection and annotation, data augmentation is a low cost way. In this paper, we propose a new method for text image augmentation. Different from traditional augmentation methods such as rotation, scaling and perspective transformation, our proposed augmentation method is designed to learn proper and efficient data augmentation which is more effective and specific for training a robust recognizer. By using a set of custom fiducial points, the proposed augmentation method is flexible and controllable. Furthermore, we bridge the gap between the isolated processes of data augmentation and network optimization by joint learning. An agent network learns from the output of the recognition network and controls the fiducial points to generate more proper training samples for the recognition network. Extensive experiments on various benchmarks, including regular scene text, irregular scene text and handwritten text, show that the proposed augmentation and the joint learning methods significantly boost the performance of the recognition networks. A general toolkit for geometric augmentation is available.


N-COMP based user identification N-COMP is a decoding strategy for noisy GT proposed by Chan et al. in #REFR .

The basic idea is to declare an item to be defective or non-defective based on the fraction of positive tests they participate in #OTHEREFR .

This significantly simplifies the run-time and storage requirements in comparison to the optimal exhaustive search strategy in Algorithm 1.

As established in Section III, the active device identification framework in non-coherent ( , k)−MnAC can be viewed as a GT problem.

This motivates us to study the performance of N-COMP approach in identifying active devices in the non-coherent ( , k)−MnAC.

Note that in the existing literature, the N-COMP GT decoder was primarily analyzed in the context of symmetric noise models wherein errors in test results are equally likely #OTHEREFR , #OTHEREFR .

Non-Coherent Active Device Identification For Massive Random Access

Abstract: Massive Machine-Type Communications (mMTC) is a key service category in the current generation of wireless networks featuring an extremely high density of energy and resource-limited devices with sparse and sporadic activity patterns. In order to enable random access in such mMTC networks, base station needs to identify the active devices while operating within stringent access delay constraints. In this paper, an energy efficient active device identification protocol is proposed in which active devices transmit On-Off Keying (OOK) modulated preambles jointly and base station employs non-coherent energy detection avoiding channel estimation overheads. The minimum number of channel-uses required by the active user identification protocol is characterized in the asymptotic regime of total number of devices when the number of active devices k scales as k = Θ(1) along with an achievability scheme relying on the equivalence of activity detection to a group testing problem. Several practical schemes based on Belief Propagation (BP) and Combinatorial Orthogonal Matching Pursuit (COMP) are also proposed. Simulation results show that BP strategies outperform COMP significantly and can operate close to the theoretical achievability bounds. In a partial-recovery setting where few misdetections are allowed, BP continues to perform well.

Non-Adaptive Probabilistic Group Testing With Noisy Measurements: Near-Optimal Bounds With Efficient Algorithms

Abstract: We consider the problem of detecting a small subset of defective items from a large set via non-adaptive “random pooling” group tests. We consider both the case when the measurements are noiseless, and the case2 when the measurements are noisy (the outcome of each group test may be independently faulty with probability q). Order-optimal results for these scenarios are known in the literature. We give information-theoretic lower bounds on the query complexity of these problems, and provide corresponding computationally efficient algorithms that match the lower bounds up to a constant factor. To the best of our knowledge this work is the first to explicitly estimate such a constant that characterizes the gap between the upper and lower bounds for these problems.


Following #REFR 15] , we use Level-3 granularity and filter out the shapes with more than 50 pairs of joints, which leaves us with 3736 chairs, 5053 tables, 719 cabinets.

Following #OTHEREFR , we use the three largest furniture categories that require realworld assembly, chairs, tables, and cabinets, and adopt the PartNet official train/validation/test split.

We use Furthest Point Sampling (FPS) to sample 1,000 points over each part mesh.

All parts are canonicalized to be zero-centered and rotated to be local axis aligned using PCA.

We detect joint points by computing all pair-wise part Chamfer Distance (eq.

3) and take the closest 50 points between two connected parts with a minimum distance of less than 0.05.

Category-Level Multi-Part Multi-Joint 3D Shape Assembly

Abstract: Shape assembly composes complex shapes geometries by arranging simple part geometries and has wide applications in autonomous robotic assembly and CAD modeling. Existing works focus on geometry reasoning and neglect the actual physical assembly process of matching and fitting joints, which are the contact surfaces connecting different parts. In this paper, we consider contacting joints for the task of multi-part assembly. A successful joint-optimized assembly needs to satisfy the bilateral objectives of shape structure and joint alignment. We propose a hierarchical graph learning approach composed of two levels of graph representation learning. The part graph takes part geometries as input to build the desired shape structure. The joint-level graph uses part joints information and focuses on matching and aligning joints. The two kinds of information are combined to achieve the bilateral objectives. Extensive experiments demonstrate that our method outperforms previous methods, achieving better shape structure and higher joint alignment accuracy.

Learning 3D Part Assembly From A Single Image

Abstract: Autonomous assembly is a crucial capability for robots in many applications. For this task, several problems such as obstacle avoidance, motion planning, and actuator control have been extensively studied in robotics. However, when it comes to task specification, the space of possibilities remains underexplored. Towards this end, we introduce a novel problem, single-image-guided 3D part assembly, along with a learningbased solution. We study this problem in the setting of furniture assembly from a given complete set of parts and a single image depicting the entire assembled object. Multiple challenges exist in this setting, including handling ambiguity among parts (e.g., slats in a chair back and leg stretchers) and 3D pose prediction for parts and part subassemblies, whether visible or occluded. We address these issues by proposing a two-module pipeline that leverages strong 2D-3D correspondences and assembly-oriented graph message-passing to infer part relationships. In experiments with a PartNet-based synthetic benchmark, we demonstrate the effectiveness of our framework as compared with three baseline approaches.


In the above, D(•, •) is a discrepancy measure between two sets of observables; for example, in the case where we are measuring the discrepancy between two density matrices, we may take D to be the Trace Distance #REFR . In Eq.

1 for the set of parameters Λ and a particular realization of the vacuum noise, followed by projection of the output and noise fields onto a desired orthonormal basis; E denotes the expectation over the vacuum noise; and the operator F computes the first-order correlations which yield the desired observable.

Given a desired observable-set, O d , describing the quantum state or any related features, our goal is to find the unknown physical parameters, Λ, that characterize the system.

We take a parameterized approach to solving the inverse problem, i.e.

Λ = Λ(θ), and solve the inverse problem by solving the optimization problem

3, by minimizing the discrepancy the model produces the properties of the optical system that would produce a result as close as possible to the desired quantum state. The inverse model is then given by

where I(•) is our inverse solver. In order to solve the optimization problem in Eq.

3, an approach based on gradient descent may be employed. The key is that the forward model of Eq.

1, while quite complicated, can be expressed in such a way that it is fully differentiable.

Designing Nonlinear Photonic Crystals For High-Dimensional Quantum State Engineering

Abstract: We propose a novel, physically-constrained and differentiable approach for the generation of D-dimensional qudit states via spontaneous parametric downconversion (SPDC) in quantum optics. We circumvent any limitations imposed by the inherently stochastic nature of the physical process and incorporate a set of stochastic dynamical equations governing its evolution under the SPDC Hamiltonian. We demonstrate the effectiveness of our model through the design of structured nonlinear photonic crystals (NLPCs) and shaped pump beams; and show, theoretically and experimentally, how to generate maximally entangled states in the spatial degree of freedom. The learning of NLPC structures offers a promising new avenue for shaping and controlling arbitrary quantum states and enables all-optical coherent control of the generated states. We believe that this approach can readily be extended from bulky crystals to thin Metasurfaces and potentially applied to other quantum systems sharing a similar Hamiltonian structures, such as superfluids and superconductors.

Trace-Distance Measure Of Coherence

Abstract: We show that trace distance measure of coherence is a strong monotone for all qubit and, so called, $X$ states. An expression for the trace distance coherence for all pure states and a semi definite program for arbitrary states is provided. We also explore the relation between $l_1$-norm and relative entropy based measures of coherence, and give a sharp inequality connecting the two. In addition, it is shown that both $l_p$-norm- and Schatten-$p$-norm-based measures violate the (strong) monotonicity for all $p\in(1,\infty)$.


As the CD classification performance in Table 4 , sharing the kernel weight in the CS-MIL with ReLU #REFR achieved better performances with a higher mean value of multiple metrics.

Inspired by #OTHEREFR and #OTHEREFR , we explores various attention mechanism designs in MIL with different activation functions and evaluate those designs on CD dataset.

We formed the CS-MIL into two strategies, differentiated by whether they shared the kernel weights while learning the embedding features from multiple scales.

Medical Image Analysis

Abstract: Analyzing high resolution whole slide images (WSIs) with regard to information across multiple scales poses a significant challenge in digital pathology. Multi-instance learning (MIL) is a common solution for working with high resolution images by classifying bags of objects (i.e. sets of smaller image patches). However, such processing is typically performed at a single scale (e.g., 20× magnification) of WSIs, disregarding the vital inter-scale information that is key to diagnoses by human pathologists. In this study, we propose a novel cross-scale MIL algorithm to explicitly aggregate inter-scale relationships into a single MIL network for pathological image diagnosis. The contribution of this paper is threefold: (1) A novel cross-scale MIL (CS-MIL) algorithm that integrates the multi-scale information and the inter-scale relationships is proposed; (2) A toy dataset with scale-specific morphological features is created and released to examine and visualize differential cross-scale attention; (3) Superior performance on both in-house and public datasets is demonstrated by our simple cross-scale MIL strategy. The official implementation is publicly available at https://github.com/hrlblab/CS-MIL.

Deep Learning Using Rectified Linear Units (Relu)

Abstract: We introduce the use of rectified linear units (ReLU) as the classification function in a deep neural network (DNN). Conventionally, ReLU is used as an activation function in DNNs, with Softmax function as their classification function. However, there have been several studies on using a classification function other than Softmax, and this study is an addition to those. We accomplish this by taking the activation of the penultimate layer $h_{n - 1}$ in a neural network, then multiply it by weight parameters $\theta$ to get the raw scores $o_{i}$. Afterwards, we threshold the raw scores $o_{i}$ by $0$, i.e. $f(o) = \max(0, o_{i})$, where $f(o)$ is the ReLU function. We provide class predictions $\hat{y}$ through argmax function, i.e. argmax $f(x)$.


In this work, EV charging demand at the station must be calculated and included as part of the power balance equation #REFR .

are the charging and discharging power for vehicle v; η v+ , η v− are the charging and discharging efficiency, respectively.

Charging and discharging can not happen at the same time.

Therefore, the set of plugged-in EVs to be charged at each time t at the station is defined as:

where t a v is the arrival time; t d v is the departure time; SoC t v is the EV battery energy level; E c v is the required charging energy of vehicle v.

Thus, assuming charging with a constant nominal charging powerP v , the overall EV demand at the railway station level can be estimated as:

An Optimal Energy Management Algorithm Considering Regenerative Braking And Renewable Energy For Ev Charging In Railway Stations

Abstract: This paper proposes a novel optimal Energy Management System (EMS) algorithm for Electric Vehicle (EV) charging in smart electric railway stations with renewable generation. As opposed to previous railway EMS methods, the proposed EMS coordinates the combined Regenerative Braking Energy (RBE), renewable generation, electric railway demand and EV charging demand at the EV parking lot of the railway station. Numerical results using a scenario-based approach on an actual railway station in Chur, Switzerland demonstrate that the proposed algorithm can effectively minimize the expected daily operating cost for the train station over an entire year. Index Terms-Electric vehicles, energy management, railway systems, regenerative braking energy, renewable energy, mixed integer linear programming.

The Effect Of The Uncertainty Of Load And Renewable Generation On The Dynamic Voltage Stability Margin

Abstract: In this paper, the impact of stochastic load and renewable generation uncertainty on the dynamic voltage stability margin is studied. Stochastic trajectories describing the uncertainty of load, wind and solar generation have been incorporated in the power system model as a set of Stochastic Differential-Algebraic Equations (SDAEs). A systematic study of Monte Carlo dynamic simulations on the IEEE 39-Bus system has been conducted to compute the stochastic load margin with all dynamic components active. Numerical results show that the uncertainty of both demand and generation may lead to a decrease on the size of the dynamic voltage stability margin, yet the variability of renewable generators may play a more significant role. Given that the integration of renewable energy will continue growing, it is of paramount importance to apply stochastic and dynamic approaches in the voltage stability study.