Network science has introduced various types of centrality measures to determine the relative importance of nodes in a network under respective circumstances #REFR .

Some results on the MPC of nonlinear systems using model reduction have been reported in the literature. For example, Wiese et al. #OTHEREFR presented an MPC method for gas turbines.

They specifically developed a lower-order internal model from a physics-based higher-order model using rigorous timescale separation arguments that can be extended to various gas turbine systems.

Zhang and Liu #OTHEREFR considered the problem of the economic model predictive control of wastewater treatment plants based on model reduction.

Their reduction is based on a technique called reduced-order trajectory segment linearization.

The authors showed via numerical simulations that, while the proposed methods lead to improved computational efficiency, they do not involve reduced control performance.

Given this context, the interplay between control and centrality has been actively investigated. Liu et al.

#OTHEREFR introduced the concept of control centrality to quantify the ability of a single node to control the entire network.

Inspired by the relationship between control centrality and the hierarchical structure in networks, the authors designed efficient attack strategies against the controllability of malicious networks. Fitch et al.

#OTHEREFR showed that the tracking performance of any leader set within a multiagent system can be quantified by a novel centrality measure called joint centrality.

For both single and multiple leaders, the authors have analytically proven the effectiveness of the centrality measure for leader selection.

Reduced-Order Model Predictive Control Of A Fish Schooling Model

Abstract: We study the problem of model predictive control (MPC) for the fish schooling model proposed by Gautrais et al. (Annales Zoologici Fennici, 2008). The high nonlinearity of the model attributed to its attraction/alignment/repulsion law suggests the need to use MPC for controlling the fish schooling's motion. However, for large schools, the hybrid nature of the law can make it numerically demanding to perform finite-horizon optimizations in MPC. Therefore, this paper proposes reducing the fish schooling model for numerically efficient MPC; the reduction is based on using the weighted average of the directions of individual fish in the school. We analytically show how using the normalized eigenvector centrality of the alignment-interaction network can yield a better reduction by comparing reduction errors. We confirm this finding on the weight and numerical efficiency of the MPC with the reduced-order model by numerical simulations. The proposed reduction allows us to control a school with up to 500 individuals. Further, we confirm that reduction with the normalized eigenvector centrality allows us to improve the control accuracy by factor of five when compared to that using constant weights.

Axioms For Centrality

Abstract: Given a social network, which of its nodes are more central? This question has been asked many times in sociology, psychology and computer science, and a whole plethora of centrality measures (a.k.a. centrality indices, or rankings) were proposed to account for the importance of the nodes of a network. In this paper, we try to provide a mathematically sound survey of the most important classic centrality measures known from the literature and propose an axiomatic approach to establish whether they are actually doing what they have been designed for. Our axioms suggest some simple, basic properties that a centrality measure should exhibit. Surprisingly, only a new simple measure based on distances, harmonic centrality, turns out to satisfy all axioms; essentially, harmonic centrality is a correction to Bavelas's classic closeness centrality designed to take unreachable nodes into account in a natural way. As a sanity check, we examine in turn each measure under the lens of information retrieval, leveraging state-of-the-art knowledge in the discipline to measure the effectiveness of the various indices in locating web pages that are relevant to a query. While there are some examples of this comparisons in the literature, here for the first time we take into consideration centrality measures based on distances, such as closeness, in an information-retrieval setting. The results match closely the data we gathered using our axiomatic approach. Our results suggest that centrality measures based on distances, which have been neglected in information retrieval in favour of spectral centrality measures in the last years, are actually of very high quality; moreover, harmonic centrality pops up as an excellent general-purpose centrality index for arbitrary directed graphs.


Complementing the work of Nayebi #REFR who reported that 87.8% of survey respondents agreed that images provide additional information compared to text, we empirically show how images are used and how essential they are to understand a question.

Results from RQ1 reveal six types of image contents (i.e., User interface, Source code, Error code, Diagram, Results, and Configuration) and four types of image sources (i.e., Desktop Application, Web Browser, Mobile Application, and Usercreated).

Furthermore, we observe that images are essential and provide complementary information compared to their associated text.

As shown in Table 2 , we find that 68% of images are essential for the questions.

For example, in this question #OTHEREFR , the developer faced a problem when using Bootstrap input groups and applying the negative margin.

To seek answers, the developer posted an image showing the unexpected user interface where the bottom highlight of the input was hidden to support the problem description.

For developers, our results provide concrete recommendations on the types of images that are commonly shared in specific situations.

Stack Overflow in general intends to be against the usage of images as raised in this post #OTHEREFR , hence our study would be used to complement the evidence-based guidelines.

For example, developers are commonly required to take a screenshot of their desktop (47%) to present a user interface (58%).

Furthermore, users tend to provide images when asking about undesired outputs in their user interface.

Meanwhile, in the aforementioned developer post, the developer expressed one of the major concerns of "links to images fail".

Understanding The Role Of Images On Stack Overflow

Abstract: Images are increasingly being shared by software developers in diverse channels including question-and-answer forums like Stack Overflow. Although prior work has pointed out that these images are meaningful and provide complementary information compared to their associated text, how images are used to support questions is empirically unknown. To address this knowledge gap, in this paper we specifically conduct an empirical study to investigate (I) the characteristics of images, (II) the extent to which images are used in different question types, and (III) the role of images on receiving answers. Our results first show that user interface is the most common image content and undesired output is the most frequent purpose for sharing images. Moreover, these images essentially facilitate the understanding of 68% of sampled questions. Second, we find that discrepancy questions are more relatively frequent compared to those without images, but there are no significant differences observed in description length in all types of questions. Third, the quantitative results statistically validate that questions with images are more likely to receive accepted answers, but do not speed up the time to receive answers. Our work demonstrates the crucial role that images play by approaching the topic from a new angle and lays the foundation for future opportunities to use images to assist in tasks like generating questions and identifying question-relatedness.

Eye Of The Mind: Image Processing For Social Coding

Abstract: Developers are increasingly sharing images in social coding environments alongside the growth in visual interactions within social networks. The analysis of the ratio between the textual and visual content of Mozilla's change requests and in Q/As of StackOverflow programming revealed a steady increase in sharing images over the past five years. Developers' shared images are meaningful and are providing complementary information compared to their associated text. Often, the shared images are essential in understanding the change requests, questions, or the responses submitted. Relying on these observations, we delve into the potential of automatic completion of textual software artifacts with visual content.


Representation Learning For Spatialmultiomniglot

In line with the finding of #REFR , increasing the number of K not only helps MDRE to reach the ground truth MI, but also the quality of representations improves from 86.7% to 100% test classification accuracy.

Figure 7b illustrates that MDRE's encoder learns representations that achieve ∼100% Omniglot character classification for both d = n 2 = 4, 9.

On the other hand, the performances of the single ratio estimator and TRE (using the same exact dimension-wise mixing to construct auxiliary distributions) both degrade as the complexity of the task increases, with TRE only reaching up to 91% and 85% for d = 4 and d = 9, respectively.

All models were trained with the same encoder architecture to ensure fair comparison.

We further studied the effect of changing K in the d = 4 setup.

For K = 1, we aggregate all the dimensionwise mixed samples into 1 class, whereas for K = 3, we separate them into their respective classes (corresponding to the number of dimensions mixed). We illustrate this effect in Figure 7c .

Estimating The Density Ratio Between Distributions With High Discrepancy Using Multinomial Logistic Regression

Abstract: Functions of the ratio of the densities p/q are widely used in machine learning to quantify the discrepancy between the two distributions p and q. For high-dimensional distributions, binary classification-based density ratio estimators have shown great promise. However, when densities are well separated, estimating the density ratio with a binary classifier is challenging. In this work, we show that the state-of-the-art density ratio estimators perform poorly on well separated cases and demonstrate that this is due to distribution shifts between training and evaluation time. We present an alternative method that leverages multi-class classification for density ratio estimation and does not suffer from distribution shift issues. The method uses a set of auxiliary densities {m k } K k=1 and trains a multi-class logistic regression to classify the samples from p, q and {m k } K k=1 into K + 2 classes. We show that if these auxiliary densities are constructed such that they overlap with p and q, then a multi-class logistic regression allows for estimating log p/q on the domain of any of the K + 2 distributions and resolves the distribution shift problems of the current state-ofthe-art methods. We compare our method to state-of-the-art density ratio estimators on both synthetic and real datasets and demonstrate its superior performance on the tasks of density ratio estimation, mutual information estimation, and representation learning. Code: https://www.blackswhan.com/mdre/

Noise Contrastive Estimation And Negative Sampling For Conditional Models: Consistency And Statistical Efficiency

Abstract: Noise Contrastive Estimation (NCE) is a powerful parameter estimation method for log-linear models, which avoids calculation of the partition function or its derivatives at each training step, a computationally demanding step in many cases. It is closely related to negative sampling methods, now widely used in NLP. This paper considers NCE-based estimation of conditional models. Conditional models are frequently encountered in practice; however there has not been a rigorous theoretical analysis of NCE in this setting, and we will argue there are subtle but important questions when generalizing NCE to the conditional case. In particular, we analyze two variants of NCE for conditional models: one based on a classification objective, the other based on a ranking objective. We show that the ranking-based variant of NCE gives consistent parameter estimates under weaker assumptions than the classification-based method; we analyze the statistical efficiency of the ranking-based and classification-based variants of NCE; finally we describe experiments on synthetic data and language modeling showing the effectiveness and trade-offs of both methods.


Through these results, we demonstrate that our approach can produce segmentation results that are comparable to the fully supervised baseline #REFR with only 0.02% annotation.

† means disabling graph propagation and relation network during inference, but note that they are still used in training. mIoU as shown in Table 4 .

This experiment also demonstrates that our method can still achieve decent performance even though the annotator ignores several objects by mistake in "One Thing One Click" scheme.

We further investigate the performance drop with a more challenging "Four Things One Click" scheme.

However, the model cannot converge well in the very first iteration due to the insufficient label and the self-training fails in this case.

Qualitative Results on ScanNet-v2 Then, we show prediction results on ScanNet-v2 in Figures 8.

See the error maps shown in (d) and (f) for better visualizations.

Ablation Studies To further study the effectiveness of selftraining, graph propagation and relation network, we conduct ablation studies on these three modules on ScanNet-v2 validation set as shown in Table 5 with single view evaluation.

"3D U-Net" indicates that the labels are propagated only based on the confidence score of the 3D U-Net itself, i.e., the unary term in Equation 2.

This ablation is designed to manifest the effectiveness of self-training.

The "3D U-Net" column in Table 5 manifests that the performance is consistently improved with self-training strategy even without pairwise energy term in Equation 2 and super-voxel partition.

One Thing One Click++: Self-Training For Weakly Supervised 3D Scene Understanding

Abstract: 3D scene understanding, e.g., point cloud semantic and instance segmentation, often requires large-scale annotated training data, but clearly, point-wise labels are too tedious to prepare. While some recent methods propose to train a 3D network with small percentages of point labels, we take the approach to an extreme and propose "One Thing One Click," meaning that the annotator only needs to label one point per object. To leverage these extremely sparse labels in network training, we design a novel self-training approach, in which we iteratively conduct the training and label propagation, facilitated by a graph propagation module. Also, we adopt a relation network to generate the per-category prototype to enhance the pseudo label quality and guide the iterative training. Besides, our model can be compatible to 3D instance segmentation equipped with a point-clustering strategy. Experimental results on both ScanNet-v2 and S3DIS show that our self-training approach, with extremely-sparse annotations, outperforms all existing weakly supervised methods for 3D semantic and instance segmentation by a large margin, and our results are also comparable to those of the fully supervised counterparts. Codes and models are available at https://github.com/liuzhengzhe/One-Thing-One-Click.

3D Semantic Segmentation With Submanifold Sparse Convolutional Networks

Abstract: Convolutional networks are the de-facto standard for analyzing spatio-temporal data such as images, videos, and 3D shapes. Whilst some of this data is naturally dense (e.g., photos), many other data sources are inherently sparse. Examples include 3D point clouds that were obtained using a LiDAR scanner or RGB-D camera. Standard "dense" implementations of convolutional networks are very inefficient when applied on such sparse data. We introduce new sparse convolutional operations that are designed to process spatially-sparse data more efficiently, and use them to develop spatially-sparse convolutional networks. We demonstrate the strong performance of the resulting models, called submanifold sparse convolutional networks (SSCNs), on two tasks involving semantic segmentation of 3D point clouds. In particular, our models outperform all prior state-of-the-art on the test set of a recent semantic segmentation competition.


We calculated average time consumption using the CIFAR-H dataset #REFR , since CIFAR-N lacks annotator time information.

We conduct a similar analysis as Section 4.2 to seek opportunities for human-machine complimentary teaming from the human difficulty perspective instead.

To quantify the human difficulty levels we use time spent labeling an image, and an entropy-based measure of human agreement.

Time Spent: Assuming the i-th human annotator spent t i (x) time (in seconds) in annotating sample x, we adopt the average time spent on sample x as a measure to indicate the difficulty level of the given task x. Mathematically, t(x) := 1 k i∈[k] t i (x).

A larget(x) means the sample x is relatively hard for human annotators since it requires humans to spend a long time on annotation.

Human agreement (entropy): Given k human annotators for a sample x, we calculate the entropy on x as:

Entropy(x) = − i∈[K] p H,i (x) • log (p H,i (x)), where p H,i (x) := 1 k j∈[k] 1(f ML,j (x) = i).

The human agreement is a metric to evaluate human consensus on a task given multiple annotations.

A higher agreement level implies less ambiguity and easier judgment.

We calculate inverse entropy using CIFAR-H because it provides 50 labels per image, enough to calculate agreement.

Do Humans And Machines Have The Same Eyes? Human-Machine Perceptual Differences On Image Classification

Abstract: Trained computer vision models are assumed to solve vision tasks by imitating human behavior learned from training labels. Most efforts in recent vision research focus on measuring the model task performance using standardized benchmarks. Limited work has been done to understand the perceptual difference between humans and machines. To fill this gap, our study first quantifies and analyzes the statistical distributions of mistakes from the two sources. We then explore human vs. machine expertise after ranking tasks by difficulty levels. Even when humans and machines have similar overall accuracies, the distribution of answers may vary. Leveraging the perceptual difference between humans and machines, we empirically demonstrate a post-hoc human-machine collaboration that outperforms humans or machines alone.

Human Uncertainty Makes Classification More Robust

Abstract: The classification performance of deep neural networks has begun to asymptote at near-perfect levels. However, their ability to generalize outside the training set and their robustness to adversarial attacks have not. In this paper, we make progress on this problem by training with full label distributions that reflect human perceptual uncertainty. We first present a new benchmark dataset which we call CIFAR10H, containing a full distribution of human labels for each image of the CIFAR10 test set. We then show that, while contemporary classifiers fail to exhibit human-like uncertainty on their own, explicit training on our dataset closes this gap, supports improved generalization to increasingly out-of-training-distribution test datasets, and confers robustness to adversarial attacks.


BERTScore #REFR assesses the similarity between two texts at the token level by measuring the soft overlap using contextual embeddings from BERT.

We select several evaluation metrics that are commonly used in summarization:

ROUGE #OTHEREFR , which is the dominant automatic evaluation metric in summarization, is widely used by researchers.

The most commonly used ROUGE measures are ROUGE-1, ROUGE-2, and ROUGE-L, which evaluate the similarity between two texts based on the overlap of unigrams, bigrams, and the longest common sequence.

Similarly, MoverScore #OTHEREFR uses n-gram embeddings that are pooled from BERT to compute the semantic distance between two texts at the n-gram level.

BARTScore #OTHEREFR 1 views evaluation as a natural language generation task and considers that when the quality of the generated text is higher, BART is more likely to generate it from the source text or the reference, or to generate the reference from it.

BARTScore can be flexibly applied to evaluate text from various perspectives.

FactCC 2 and DAE 3 are two factuality metrics based on classification.

When evaluating a summary, we use NLTK 4 to split it into individual sentences and classify each one as factually correct or not.

Human-Like Summarization Evaluation With Chatgpt

Abstract: Evaluating text summarization is a challenging problem, and existing evaluation metrics are far from satisfactory. In this study, we explored ChatGPT's ability to perform humanlike summarization evaluation using four human evaluation methods on five datasets. We found that ChatGPT was able to complete annotations relatively smoothly using Likert scale scoring, pairwise comparison, Pyramid, and binary factuality evaluation. Additionally, it outperformed commonly used automatic evaluation metrics on some datasets. Furthermore, we discussed the impact of different prompts, compared its performance with that of human evaluation, and analyzed the generated explanations and invalid responses.

Bertscore: Evaluating Text Generation With Bert

Abstract: We propose BERTScore, an automatic evaluation metric for text generation. Analogously to common metrics, BERTScore computes a similarity score for each token in the candidate sentence with each token in the reference sentence. However, instead of exact matches, we compute token similarity using contextual embeddings. We evaluate using the outputs of 363 machine translation and image captioning systems. BERTScore correlates better with human judgments and provides stronger model selection performance than existing metrics. Finally, we use an adversarial paraphrase detection task and show that BERTScore is more robust to challenging examples compared to existing metrics.


Following the same paradigm used in #REFR , we construct queries for each purchased item with category information.

Besides, we selected three categories with different sizes: Clothing, Shoes & Jewelry, Toys & Games, Electronics.

These datasets both contain several categories so that users may have different interests.

Following the strategy in References #OTHEREFR , we extracted the users' product purchasing behaviors based on their reviews, i.e., the products they reviewed are the ones they purchased.

Our model uses the previously purchased products in a neighboring window size to model the short-term user interests.

We further filtered the dataset to make sure each user has at least 20 purchased products (i.e., 20 reviews).

This strategy is based on the finding that directed product search is users' search for a producer's name, a brand, or a set of terms describing product category.

We partitioned each of the four datasets into three sets: training, validation and testing sets.

We first extracted the userproduct pairs from users' reviews, and then extracted the queries for these products, getting triplets.

For each dataset, the last purchasing transaction of each user is held for the testing set, the second last for the validation set, and the rest for the training set.

Moreover, we hid the reviews of the validation and testing sets in the training phase to simulate the real-world scenarios.

Hierarchically Fusing Long And Short-Term User Interests For Click-Through Rate Prediction In Product Search

Abstract: Estimating Click-Through Rate (CTR) is a vital yet challenging task in personalized product search. However, existing CTR methods still struggle in the product search settings due to the following three challenges including how to more effectively extract users' short-term interests with respect to multiple aspects, how to extract and fuse users' long-term interest with short-term interests, how to address the entangling characteristic of long and short-term interests. To resolve these challenges, in this paper, we propose a new approach named Hierarchical Interests Fusing Network (HIFN), which consists of four basic modules namely Short-term Interests Extractor (SIE), Long-term Interests Extractor (LIE), Interests Fusion Module (IFM) and Interests Disentanglement Module (IDM). Specifically, SIE is proposed to extract user's short-term interests by integrating three fundamental interests encoders within it namely query-dependent, target-dependent and causal-dependent interest encoder, respectively, followed by delivering the resultant representation to the module LIE, where it can effectively capture user longterm interests by devising an attention mechanism with respect to the short-term interests from SIE module. In IFM, the achieved long and short-term interests are further fused in an adaptive manner, followed by concatenating it with original raw context features for the final prediction result. Last but not least, considering the entangling characteristic of long and short-term interests, IDM further devises a self-supervised framework to disentangle long and short-term interests. Extensive offline and online evaluations on a real-world e-commerce platform demonstrate the superiority of HIFN over state-of-the-art methods.

Attentive Long Short-Term Preference Modeling For Personalized Product Search

Abstract: E-commerce users may expect different products even for the same query, due to their diverse personal preferences. It is well-known that there are two types of preferences: long-term ones and short-term ones. The former refers to user' inherent purchasing bias and evolves slowly. By contrast, the latter reflects users' purchasing inclination in a relatively short period. They both affect users' current purchasing intentions. However, few research efforts have been dedicated to jointly model them for the personalized product search. To this end, we propose a novel Attentive Long Short-Term Preference model, dubbed as ALSTP, for personalized product search. Our model adopts the neural networks approach to learn and integrate the long- and short-term user preferences with the current query for the personalized product search. In particular, two attention networks are designed to distinguish which factors in the short-term as well as long-term user preferences are more relevant to the current query. This unique design enables our model to capture users' current search intentions more accurately. Our work is the first to apply attention mechanisms to integrate both long- and short-term user preferences with the given query for the personalized search. Extensive experiments over four Amazon product datasets show that our model significantly outperforms several state-of-the-art product search methods in terms of different evaluation metrics.


DistilBERT, as a transformerbased model, is better in capturing long-term dependencies in an input sequence #REFR .

Recently, it was shown that ELMo and BERT make no significant difference in semantic analysis #OTHEREFR .

Here it is observed that although they are close-by in the null context, DistilBERT is more robust than ELMo in the cross-context in text classification.

The findings of this study are in line with prior work.

The fairly comparable scores of ELMo and the traditional baselines in the null context supports the observation of #OTHEREFR that is, when it comes to contextual embeddings, there is only a small improvement in learning semantics over traditional ML methods.

DistilBERT is on par with or exceeding ELMo on a binary text classification task #OTHEREFR .

DistilBERT is lighter than ELMo and has a shorter training time #OTHEREFR .

Here it should be noted that the experimental settings of the previous work and In this study, ELMo and DistilBERT are compared on their fine-tuning performance on two binary text classification tasks.

The main focus was to see how much can these models be benefited in a practical way without any modification to the pretraining outputs.

But the models were actually pretrained on entirely different corpora (ELMo on One Billion Words Benchmark #OTHEREFR , DistilBERT on English Wikipedia and Toronto BookCorpus #OTHEREFR ).

If the models were also pretrained from scratch on the same corpus, it would be ensured that they utilize the same knowledge to learn the context.

Abstract: I am grateful to my family for their unconditional love and patience. I am grateful to ArzucanÖzgür, for being such an inspiring figure by her selfless devotion to research the most righteous way with the passion to contribute to the community. I am grateful to Ali Hürriyetoglu, for being such a role model, who could somehow always find a way to turn the mist of research questions into a structured path to create practical solutions by combining creativity and technique. I cannot thank enough my dear friends who put up with my whims throughout this journey. I thank fellows from TabiLAB for inspiring me with their brilliance, invaluable insights and recommendations. I thank Koç University EMW research team for their generosity in sharing the data which was created with blood, sweat and tears. I feel lucky that I got to meet fellows in EMW project engineering team who invested their precious time and energy to support me in this study from the very beginning. Lastly, I owe the deepest gratitude to our professors and staff members in our department who taught us how to form such a great community and made it feel like the dearest home from the day one.

Attention Is All You Need

Abstract: The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.


It is worth noting that in the literature #REFR , it was emphasized that some unreliable results may be generated during the application of the CycleGAN, resulting in misdiagnosis of the disease. In order to solve this problem, Hiasa et al.

However, in most cases, it is very difficult to fully obtain the paired medical images, so the monitoring scheme has certain limitations.

Consequently, unsupervised CycleGAN has a broader application prospect in the field of multimodal medical imaging generation.

The model can find the mapping relationship between the source image and the target image, which can effectively eliminate the limitation of a pair of corresponding image pairs in the training process.

In addition, the CycleGAN also uses cycle consistency loss to preserve key attributes between the input image and the generated image. For example, based on the CycleGAN, Wolterink et al.

realized the direct synthesis of brain CT images from brain MR images without paired data #OTHEREFR .

improved the CycleGAN network structure by adding gradient consistency loss during the training, then the accuracy of identifying boundaries is raised #OTHEREFR . While Lilian et al.

improved the loss function to solve the existed problem that the gradient disappearance caused by the non-coincidence of the true sample and the generated sample is difficult to train #OTHEREFR However, there still exist the problems in the CycleGAN, such as long training time, slow convergence speed and easy to ignore spatial location information.

In recent years, generative adversarial networks (GANs) #OTHEREFR have been proved to be a promising medical imaging generation method #OTHEREFR .

GANs have shown excellent performance in the field of medical images, including medical imaging reconstruction #OTHEREFR , medical imaging classification #OTHEREFR , medical imaging detection #OTHEREFR , medical imaging segmentation #OTHEREFR and medical imaging denoising #OTHEREFR .

Multimodal medical imaging generation obtains the mapping function from the source image to the target image by learning.

Two-Stage Mr Image Segmentation Method For Brain Tumors Based On Attention Mechanism

Abstract: Multimodal magnetic resonance imaging (MRI) can reveal different patterns of human tissue and is crucial for clinical diagnosis. However, limited by cost, noise and manual labeling, obtaining diverse and reliable multimodal MR images remains a challenge. For the same lesion, different MRI manifestations have great differences in background information, coarse positioning and fine structure. In order to obtain better generation and segmentation performance, a coordination-spatial attention generation adversarial network (CASP-GAN) based on the cycle-consistent generative adversarial network (CycleGAN) is proposed. The performance of the generator is optimized by introducing the Coordinate Attention (CA) module and the Spatial Attention (SA) module. The two modules can make full use of the captured location information, accurately locating the interested region, and enhancing the generator model network structure. The ability to extract the structure information and the detailed information of the original medical image can help generate the desired image with higher quality. There exist some problems in the original CycleGAN that the training time is long, the parameter amount is too large, and it is difficult to converge. In response to this problem, we introduce the Coordinate Attention (CA) module to replace the Res Block to reduce the number of parameters, and cooperate with the spatial information extraction network above to strengthen the information extraction ability. On the basis of CASP-GAN, an attentional generative cross-modality segmentation (AGCMS) method is further proposed. This method inputs the modalities generated by CASP-GAN and the real modalities into the segmentation network for brain tumor segmentation. Experimental results show that CASP-GAN outperforms CycleGAN and some state-of-the-art methods in PSNR, SSMI and RMSE in most tasks. In addition, the Dice and Hausdorff95 obtained by AGCMS segmentation are higher than the values corresponding to a single modality, and are close to the values obtained by multiple real modalities, indicating that the method can achieve similar effects as multi-modalities.. In summary, the method proposed in this paper can be used as an effective method for clinical diagnosis of brain tumors and has broad application prospects.

Distribution Matching Losses Can Hallucinate Features In Medical Image Translation

Abstract: This paper discusses how distribution matching losses, such as those used in CycleGAN, when used to synthesize medical images can lead to mis-diagnosis of medical conditions. It seems appealing to use these new image synthesis methods for translating images from a source to a target domain because they can produce high quality images and some even do not require paired data. However, the basis of how these image translation models work is through matching the translation output to the distribution of the target domain. This can cause an issue when the data provided in the target domain has an over or under representation of some classes (e.g. healthy or sick). When the output of an algorithm is a transformed image there are uncertainties whether all known and unknown class labels have been preserved or changed. Therefore, we recommend that these translated images should not be used for direct interpretation (e.g. by doctors) because they may lead to misdiagnosis of patients based on hallucinated image features by an algorithm that matches a distribution. However there are many recent papers that seem as though this is the goal.


B. Topics Regarding Emerging Ml Techniques

In response, graph neural networks (GNNs), the deep learning models that are capable of tackling graph-related tasks, were proposed #REFR .

This is particularly problematic when a grid corresponds to a large region in the real world.

Instead, we can use graph models for ride-hailing planning #OTHEREFR , #OTHEREFR . Road networks can be modeled as graphs.

In a road graph, nodes are geographical locations, and edges are travel paths.

Additional information, such as the number of riders and drivers, and the travel costs on paths, can be used as representations of the nodes and edges.

Such graphs are represented in a non-Euclidean space which is difficult for knowledge extraction using traditional ML techniques.

We propose to use GNN to solve the ride-hailing planning problem in an end-to-end manner.

Specifically, we can model road networks via graphs, design a GNN-based model to extract features from the graphs, and then map them to planning decisions.

There are existing works that have employed GNNs in the transportation area, e.g., to calculate traffic predictions #OTHEREFR , #OTHEREFR and demand predictions #OTHEREFR , #OTHEREFR .

Only a limited number of works however have used GNN in ride-hailing planning, e.g., #OTHEREFR , #OTHEREFR .

The algorithm in #OTHEREFR can only match drivers and riders that are located on the same road segment, which restricts many other drivers and riders from being matched, and thus could lead to poor performance of their proposed method in terms of measures such as the total waiting times of riders.

A Survey Of Machine Learning-Based Ride-Hailing Planning

Abstract: Ride-hailing is a sustainable transportation paradigm where riders access door-to-door traveling services through a mobile phone application, which has attracted a colossal amount of usage. There are two major planning tasks in a ride-hailing system: (1) matching, i.e., assigning available vehicles to pick up the riders, and (2) repositioning, i.e., proactively relocating vehicles to certain locations to balance the supply and demand of ride-hailing services. Recently, many studies of ride-hailing planning that leverage machine learning techniques have emerged. In this article, we present a comprehensive overview on latest developments of machine learning-based ride-hailing planning. To offer a clear and structured review, we introduce a taxonomy into which we carefully fit the different categories of related works according to the types of their planning tasks and solution schemes, which include collective matching, distributed matching, collective repositioning, distributed repositioning, and joint matching and repositioning. We further shed light on many real-world datasets and simulators that are indispensable for empirical studies on machine learning-based ride-hailing planning strategies. At last, we propose several promising research directions for this rapidly growing research and practical field.

A Comprehensive Survey On Graph Neural Networks

Abstract: Deep learning has revolutionized many machine learning tasks in recent years, ranging from image classification and video processing to speech recognition and natural language understanding. The data in these tasks are typically represented in the Euclidean space. However, there is an increasing number of applications, where data are generated from non-Euclidean domains and are represented as graphs with complex relationships and interdependency between objects. The complexity of graph data has imposed significant challenges on the existing machine learning algorithms. Recently, many studies on extending deep learning approaches for graph data have emerged. In this article, we provide a comprehensive overview of graph neural networks (GNNs) in data mining and machine learning fields. We propose a new taxonomy to divide the state-of-the-art GNNs into four categories, namely, recurrent GNNs, convolutional GNNs, graph autoencoders, and spatial-temporal GNNs. We further discuss the applications of GNNs across various domains and summarize the open-source codes, benchmark data sets, and model evaluation of GNNs. Finally, we propose potential research directions in this rapidly growing field.

graph-related tasks
graph neural networks


The order of preferences is given by the strictly ranked rate utilities of the two sides #REFR .

In this section, we propose assigning/matching the countermeasures to the attacks by a framework that considers stability as the solution concept instead of optimality. The applied framework involves a two-sided matching game.

A Stable Matching Problem (SMP) is produced by a distributed process that matches together preference relations of the two sides that are of the same size.

SM solutions have been broadly used in wireless networks for problem-solving #OTHEREFR .

In our problem, however, the number of detected attacks might be different from the number of countermeasures (i.e., different set sizes), which means we need to seek a many-to-one generalization of SMP called the HR problem #OTHEREFR .

Joint Security-Vs-Qos Game Theoretical Optimization For Intrusion Response Mechanisms For Future Network Systems

Abstract: Network connectivity exposes the network infrastructure and assets to vulnerabilities that attackers can exploit. Protecting network assets against attacks requires the application of security countermeasures. Nevertheless, employing countermeasures incurs costs, such as monetary costs, along with time and energy to prepare and deploy the countermeasures. Thus, an Intrusion Response System (IRS) shall consider security and QoS costs when dynamically selecting the countermeasures to address the detected attacks. This has motivated us to formulate a joint Security-vs-QoS optimization problem to select the best countermeasures in an IRS. The problem is then transformed into a matching game-theoretical model. Considering the monetary costs and attack coverage constraints, we first derive the theoretical upper bound for the problem and later propose stable matching-based solutions to address the trade-off. The performance of the proposed solution, considering different settings, is validated over a series of simulations.

Distributed Channel Assignment In Cognitive Radio Networks: Stable Matching And Walrasian Equilibrium

Abstract: We consider a set of secondary transmitter-receiver pairs in a cognitive radio setting. Based on channel sensing and access performances, we consider the problem of assigning channels orthogonally to secondary users through distributed coordination and cooperation algorithms. Two economic models are applied for this purpose: matching markets and competitive markets. In the matching market model, secondary users and channels build two agent sets. We implement a stable matching algorithm in which each secondary user, based on his achievable rate, proposes to the coordinator to be matched with desirable channels. The coordinator accepts or rejects the proposals based on the channel preferences which depend on interference from the secondary user. The coordination algorithm is of low complexity and can adapt to network dynamics. In the competitive market model, channels are associated with prices and secondary users are endowed with monetary budget. Each secondary user, based on his utility function and current channel prices, demands a set of channels. A Walrasian equilibrium maximizes the sum utility and equates the channel demand to their supply. We prove the existence of Walrasian equilibrium and propose a cooperative mechanism to reach it. The performance and complexity of the proposed solutions are illustrated by numerical simulations.


Besides, MANS performs particularly well in Hit@1 and MRR, which are sensitive to high-rank results #REFR .

We could observe that existing NS methods have poor performance and they are even worse than the normal NS.

According to our previous analysis, these NS methods are designed for general KGE models and are unsuitable for the multi-modal scenario where modal information is carefully considered.

They could not align different embeddings of each entity and get bad performance in MMKGE. The outperformance of MANS.

MANS could achieve better link prediction results compared with baselines.

For example, MANS-A achieves much better Hit@1 on FB15K compared with baselines (from 0.318 to 0.353, a relative improvement of 9.9%).

This means that MANS can largely improve the accurate discriminatory ability of the model by aligning structural and visual embeddings. Necessity and effectiveness of MANS-V.

According to the previous section, MANS-V is designed to align different modal information.

Though it does not perform better than baseline methods, MANS-V is the fundamental component of the other three settings of MANS.

Besides, we could prove with such a result that both modal alignment and positivenegative discrimination are important for MMKGE, which could be achieved by MANS-V and normal NS respectively.

MANS-T, MANS-H, and MANS-A could perform better because they combine the advantages of both. In summary, MANS-V is a necessary design for MMKGE. Comparison of different MANS settings.

Modality-Aware Negative Sampling For Multi-Modal Knowledge Graph Embedding

Abstract: Negative sampling (NS) is widely used in knowledge graph embedding (KGE), which aims to generate negative triples to make a positive-negative contrast during training. However, existing NS methods are unsuitable when multi-modal information is considered in KGE models. They are also inefficient due to their complex design. In this paper, we propose Modality-Aware Negative Sampling (MANS) for multi-modal knowledge graph embedding (MMKGE) to address the mentioned problems. MANS could align structural and visual embeddings for entities in KGs and learn meaningful embeddings to perform better in multi-modal KGE while keeping lightweight and efficient. Empirical results on two benchmarks demonstrate that MANS outperforms existing NS methods. Meanwhile, we make further explorations about MANS to confirm its effectiveness.

Nscaching: Simple And Efficient Negative Sampling For Knowledge Graph Embedding

Abstract: Knowledge graph (KG) embedding is a fundamental problem in data mining research with many real-world applications. It aims to encode the entities and relations in the graph into low dimensional vector space, which can be used for subsequent algorithms. Negative sampling, which samples negative triplets from non-observed ones in the training data, is an important step in KG embedding. Recently, generative adversarial network (GAN), has been introduced in negative sampling. By sampling negative triplets with large scores, these methods avoid the problem of vanishing gradient and thus obtain better performance. However, using GAN makes the original model more complex and harder to train, where reinforcement learning must be used. In this paper, motivated by the observation that negative triplets with large scores are important but rare, we propose to directly keep track of them with cache. However, how to sample from and update the cache are two important questions. We carefully design the solutions, which are not only efficient but also achieve good balance between exploration and exploitation. In this way, our method acts as a "distilled" version of previous GAN-based methods, which does not waste training time on additional parameters to fit the full distribution of negative triplets. The extensive experiments show that our method can gain significant improvement on various KG embedding models, and outperform the state-of-the-arts negative sampling methods based on GAN.


Hierarchical agglomerative clustering algorithms (e.g., #REFR ) allow for partitioning the datasets for which merely a pairwise distance function (e.g., a metric) is defined.

Cluster analysis (e.g., #OTHEREFR ) is a machine learning technique where we discover interesting or otherwise useful partitions of a given dataset in a purely unsupervised way.

Most importantly, the number of clusters is not set in advance -a whole hierarchy of nested partitions can be generated with ease and then depicted on a tree-like diagram called a dendrogram.

Hierarchical agglomerative clustering evolves around one simple idea: in each step, we merge the pair of closest clusters.

To measure the proximity between two point sets, the intracluster distance is defined as an extension of the point-pairwise distance called a linkage function.

For instance, in the single linkage approach, the distance between a cluster pair is given by the distance between the closest pair of points, one from the first cluster, the other from the second one.

In complete linkage, on the other hand, we take the farthest-away pair.

Hierarchical Clustering With Owa-Based Linkages, The Lance-Williams Formula, And Dendrogram Inversions

Abstract: Agglomerative hierarchical clustering based on Ordered Weighted Averaging (OWA) operators not only generalises the single, complete, and average linkages, but also includes intercluster distances based on a few nearest or farthest neighbours, trimmed and winsorised means of pairwise point similarities, amongst many others. We explore the relationships between the famous Lance-Williams update formula and the extended OWA-based linkages with weights generated via infinite coefficient sequences. Furthermore, we provide some conditions for the weight generators to guarantee the resulting dendrograms to be free from unaesthetic inversions.

Modern Hierarchical, Agglomerative Clustering Algorithms

Abstract: This paper presents algorithms for hierarchical, agglomerative clustering which perform most efficiently in the general-purpose setup that is given in modern standard software. Requirements are: (1) the input data is given by pairwise dissimilarities between data points, but extensions to vector data are also discussed (2) the output is a "stepwise dendrogram", a data structure which is shared by all implementations in current standard software. We present algorithms (old and new) which perform clustering in this setting efficiently, both in an asymptotic worst-case analysis and from a practical point of view. The main contributions of this paper are: (1) We present a new algorithm which is suitable for any distance update scheme and performs significantly better than the existing algorithms. (2) We prove the correctness of two algorithms by Rohlf and Murtagh, which is necessary in each case for different reasons. (3) We give well-founded recommendations for the best current algorithms for the various agglomerative clustering schemes.


Although it has been demonstrated that adversarial patches are quite powerful at causing misclassi cation, these patches are highlighted using standard network interpretation methods, thereby disclosing the identity of the adversary #REFR .

One of the most well-known network interpretation algorithms, Grad-CAM #OTHEREFR , outperforms other state-of-theart interpretation algorithms on a sanity check. e Grad-CAM visualization results for traditional adversarial patch vs.

AdvRain are evaluated using an ImageNet pretrained VGG-19 classi er (adding low vs high frequency pa erns).

Unlike patch-based a acks that shi the model's focus from the object to the location of the patch, making them detectable, our a ack causes the model to overlook some important features that help the model make the decision, as shown in Figure 11 .

Advrain: Adversarial Raindrops To A Ack Camera-Based Smart Vision Systems

Abstract: Vision-based perception modules are increasingly deployed in many applications, especially autonomous vehicles and intelligent robots. ese modules are being used to acquire information about the surroundings and identify obstacles. Hence, accurate detection and classi cation are essential to reach appropriate decisions and take appropriate and safe actions at all times. Current studies have demonstrated that "printed adversarial attacks", known as physical adversarial attacks, can successfully mislead perception models such as object detectors and image classi ers. However, most of these physical attacks are based on noticeable and eye-catching patterns for generated perturbations making them identi able/detectable by human eye or in test drives. In this paper, we propose a camera-based inconspicuous adversarial attack (AdvRain) capable of fooling camera-based perception systems over all objects of the same class. Unlike mask based fake-weather attacks that require access to the underlying computing hardware or image memory, our attack is based on emulating the e ects of a natural weather condition (i.e., Raindrops) that can be printed on a translucent sticker, which is externally placed over the lens of a camera. Note, such perturbations are still inconspicuous in real-world deployments and their presence goes unnoticed due to their association with a natural phenomenon (as also advocated in [1]). To accomplish this, we provide an iterative process based on performing a random search aiming to identify critical positions to make sure that the performed transformation is adversarial for a target classi er. Our transformation is based on blurring prede ned parts of the captured image corresponding to the areas covered by the raindrop. We achieve a drop in average model accuracy of more than 45% and 40% on VGG19 for ImageNet and Resnet34 for Caltech-101, respectively, using only 20 raindrops. Index Terms-Adversarial machine learning, physical adversarial attack, Security, e ciency, perturbations, physical attacks. I. I e emergence of deep learning (DL) is creating disruptive transformations in a wide range of sectors especially autonomous driving [2]. For instance, leading manufacturers such as Google, Audi, BMW, and Tesla are striving to create autonomous vehicles (AVs) by combining this cu ing-edge technology with low-cost cameras forming the vision-based perception modules. AVs are being increasingly equipped with these modules to address high-pressure real-life scenarios, reach suitable decisions, and take appropriate and safe actions. In fact, their incorporation increased product demand and helped the market of autonomous vehicles grow. According to the Strategic Market Research (SMR), the market for autonomous vehicles will reach $196.97 billion by 2030 [3], growing at a CAGR of 25.7%.

Towards Hiding Adversarial Examples From Network Interpretation

Abstract: This work was performed under ::: the following financial assistance award: 60NANB18D279 ::: from U.S. Department of Commerce, National Institute of ::: Standards and Technology, and also funding from SAP SE


Such interpretations can help improve the transparency and trustworthiness of the models, as well as identify potential sources of bias or error #REFR .

Since the UPD annotation in this dataset is based on a single factor, it cannot be utilized for UPD detection with complex scenarios and UPD factor ranking analysis.

Consequently, we construct our experimental data using the Place Pulse 2.0 dataset in this work.

Interpretability of Deep Learning Models.

Although deep learning techniques have shown significant potential to achieve high accuracy for various complex tasks, it is crucial to understand how the model reaches its conclusions.

This has led to a growing interest in developing methods for interpreting deep learning models, with the goal of understanding the reasoning behind their predictions.

Visual explanations are a class of methods for interpreting deep learning models that use visualizations to highlight the features of the input data that are most important for the model's decision #OTHEREFR .

These methods offer a clear understanding of how the model processes the input data and which features are responsible for influencing the output.

Visual explanations can also provide guidance to identify instances where the model may be making errors or exhibiting bias.

There are various types of visual explanation for deep learning models, such as attention-based maps, activation maps, and occlusion-based methods #OTHEREFR .

Attention maps #OTHEREFR highlight the regions of an image that are most important to the decision of the model by computing the gradients of the output with respect to the input.

Updexplainer: An Interpretable Transformer-Based Framework For Urban Physical Disorder Detection Using Street View Imagery

Abstract: Urban Physical Disorder (UPD), such as old or abandoned buildings, broken sidewalks, litter, and graffiti, has a negative impact on residents' quality of life. They can also increase crime rates, cause social disorder, and pose a public health risk. Currently, there is a lack of efficient and reliable methods for detecting and understanding UPD. To bridge this gap, we propose UPDExplainer, an interpretable transformer-based framework for UPD detection. We first develop a UPD detection model based on the Swin Transformer architecture, which leverages readily accessible street view images to learn discriminative representations. In order to provide clear and comprehensible evidence and analysis, we subsequently introduce a UPD factor identification and ranking module that combines visual explanation maps with semantic segmentation maps. This novel integrated approach enables us to identify the exact objects within street view images that are responsible for physical disorders and gain insights into the underlying causes. Experimental results on the re-annotated Place Pulse 2.0 dataset demonstrate promising detection performance

Grad-Cam: Visual Explanations From Deep Networks Via Gradient-Based Localization

Abstract: We propose a technique for producing "visual explanations" for decisions from a large class of CNN-based models, making them more transparent. Our approach - Gradient-weighted Class Activation Mapping (Grad-CAM), uses the gradients of any target concept, flowing into the final convolutional layer to produce a coarse localization map highlighting the important regions in the image for predicting the concept. Unlike previous approaches, GradCAM is applicable to a wide variety of CNN model-families: (1) CNNs with fully-connected layers (e.g. VGG), (2) CNNs used for structured outputs (e.g. captioning), (3) CNNs used in tasks with multimodal inputs (e.g. VQA) or reinforcement learning, without any architectural changes or re-training. We combine GradCAM with fine-grained visualizations to create a high-resolution class-discriminative visualization and apply it to off-the-shelf image classification, captioning, and visual question answering (VQA) models, including ResNet-based architectures. In the context of image classification models, our visualizations (a) lend insights into their failure modes (showing that seemingly unreasonable predictions have reasonable explanations), (b) are robust to adversarial images, (c) outperform previous methods on weakly-supervised localization, (d) are more faithful to the underlying model and (e) help achieve generalization by identifying dataset bias. For captioning and VQA, our visualizations show that even non-attention based models can localize inputs. Finally, we conduct human studies to measure if GradCAM explanations help users establish trust in predictions from deep networks and show that GradCAM helps untrained users successfully discern a "stronger" deep network from a "weaker" one. Our code is available at this https URL. A demo and a video of the demo can be found at this http URL and this http URL.


As illustrated in MoCoGAN #REFR , the latent space of GANs can be decomposed into content subspace and motion subspace.

That results in more complex networks, larger model sizes, and training costs.

Early works #OTHEREFR - #OTHEREFR attempt to generate temporally coherent videos from random noises with conv-based GAN #OTHEREFR models directly, leading to high computational costs and unsatisfying performance on large-resolution datasets.

Video synthesis frameworks based on pre-trained image generators are proposed to pursue higher quality and larger resolution, including MoCoGAN-HD and StyleVideoGAN.

They design motion generators to manipulate latent codes for synthesizing videos based on pre-trained image generators such as BigGAN #OTHEREFR and StyleGAN2 #OTHEREFR .

StyleGAN-V #OTHEREFR proposes to generate videos by concatenating sequences of encoded motion codes to the constant input tensor of StyleGAN2.

Following prior works, we focus on studying modern video generation datasets, where frames in a video share the same contents but vary in motions, e.g., persons talking in FaceForensics 256 2 #OTHEREFR and cloud moving in SkyTimelapse 256 2 #OTHEREFR .

Previous works #OTHEREFR , #OTHEREFR propose sequential generation methods to synthesize videos with image-based generators.

Additional motion code generators or randomly sampled motion codes are needed for motion generation.

They train generative models to fit the distribution of videos without unique methods of keeping contents consistent. The contents and motions are learned implicitly during training.

We find that such methods result in inappropriate content editing when generating motions.

Motionvideogan: A Novel Video Generator Based On The Motion Space Learned From Image Pairs

Abstract: Video generation has achieved rapid progress benefiting from high-quality renderings provided by powerful image generators. We regard the video synthesis task as generating a sequence of images sharing the same contents but varying in motions. However, most previous video synthesis frameworks based on pre-trained image generators treat content and motion generation separately, leading to unrealistic generated videos. Therefore, we design a novel framework to build the motion space, aiming to achieve content consistency and fast convergence for video generation. We present MotionVideoGAN, a novel video generator synthesizing videos based on the motion space learned by pre-trained image pair generators. Firstly, we propose an image pair generator named MotionStyleGAN to generate image pairs sharing the same contents and producing various motions. Then we manage to acquire motion codes to edit one image in the generated image pairs and keep the other unchanged. The motion codes help us edit images within the motion space since the edited image shares the same contents with the other unchanged one in image pairs. Finally, we introduce a latent code generator to produce latent code sequences using motion codes for video generation. Our approach achieves state-of-the-art performance on the most complex video dataset ever used for unconditional video generation evaluation, UCF101. The source code is available on https://github.com/bbzhu-jy16/MotionVideoGAN.

Mocogan: Decomposing Motion And Content For Video Generation

Abstract: Visual signals in a video can be divided into content and motion. While content specifies which objects are in the video, motion describes their dynamics. Based on this prior, we propose the Motion and Content decomposed Generative Adversarial Network (MoCoGAN) framework for video generation. The proposed framework generates a video by mapping a sequence of random vectors to a sequence of video frames. Each random vector consists of a content part and a motion part. While the content part is kept fixed, the motion part is realized as a stochastic process. To learn motion and content decomposition in an unsupervised manner, we introduce a novel adversarial learning scheme utilizing both image and video discriminators. Extensive experimental results on several challenging datasets with qualitative and quantitative comparison to the state-of-the-art approaches, verify effectiveness of the proposed framework. In addition, we show that MoCoGAN allows one to generate videos with same content but different motion as well as videos with different content and same motion.


The label token z is mapped into the predicted label y by a verbalizer #REFR to complete the downstream tasks.

The intuition of prompting is to convert the downstream tasks into a similar form as the pre-training stage.

For pre-trained models whose pretraining objective is to predict the next token given previous tokens, e.g., GPT-3 #OTHEREFR and BLOOM #OTHEREFR , prompting aims to ask a model to predict the next token (i.e.

"correct" or "wrong" in this task) given previous tokens (patch contents and demonstrations).

To help pre-trained models understand task-specific information, prompting modifies the input data by adding a piece of text description, namely prompt templates.

The prompt template is a textual string that has two slots: (1) an input slot [X] for original input data x and (2) an answer slot [Z] for the predicted answer/label token z.

The verbalizer, denoted as V , is a function that maps each predicted label token z to a classŷ in the target class set Y :

In the APCA task, the label token set Z includes two tokens, i.e., {"correct", "wrong"}, and the class set Y contains {-, +} for indicating correct (clean) and wrong (overfitting) patches, respectively.

Note that the verbalizer is manually defined instead of learned from data.

Patchzero: Zero-Shot Automatic Patch Correctness Assessment

Abstract: Automated Program Repair (APR) techniques have shown more and more promising results in fixing real-world bugs. Despite the effectiveness, APR techniques still face an overfitting problem: a generated patch can be incorrect although it passes all tests. It is time-consuming to manually evaluate the correctness of generated patches that can pass all tests. To address this problem, many approaches have been proposed to automatically assess the correctness of patches generated by APR techniques. However, existing approaches require a large set of manually labeled patches as the training data. To mitigate the issue, in this study, we propose PatchZero, the patch correctness assessment by adopting large pre-trained models. Specifically, for patches generated by a new or unseen APR tool, PatchZero does not need labeled patches of this new or unseen APR tool for training (i.e., zero-shot) but directly queries the large pre-trained model to get predictions on the correctness labels without training. In this way, PatchZero can reduce the manual labeling effort when building a model to automatically assess the correctness of generated patches of new APR tools. To provide knowledge regarding the automatic patch correctness assessment (APCA) task to the large pre-trained models, we also design an instance-wise demonstration formation strategy by using contrastive learning. Specifically, PatchZero selects semantically similar patches to help the large pre-trained model to give more accurate predictions on the unlabeled patches. Our experimental results showed that PatchZero can achieve an accuracy of 82.7% and an F1-score of 86.0% on average although no labeled patch of the new or unseen APR tool is available. In addition, our proposed technique outperformed the prior state-of-the-art by a large margin.

Exploiting Cloze Questions For Few-Shot Text Classification And Natural Language Inference.

Abstract: Some NLP tasks can be solved in a fully unsupervised fashion by providing a pretrained language model with "task descriptions" in natural language (e.g., Radford et al., 2019). While this approach underperforms its supervised counterpart, we show in this work that the two ideas can be combined: We introduce Pattern-Exploiting Training (PET), a semi-supervised training procedure that reformulates input examples as cloze-style phrases which help the language model understand the given task. Theses phrases are then used to assign soft labels to a large set of unlabeled examples. Finally, regular supervised training is performed on the resulting training set. On several tasks, we show that PET outperforms both supervised training and unsupervised approaches in low-resource settings by a large margin.


In #REFR , the authors discussed power and time allocation in NOMA-assisted MEC and derived closed-form expressions for optimal MEC offloading policies.

In #OTHEREFR , the authors studied a IoT MEC network with multiple users and edge servers, where users randomly upload various tasks to the edge servers, and the servers utilize shared computation resources to process the uploaded tasks.

To optimize the resource utilization, the authors proposed a heuristic resource scheduling policy.

In #OTHEREFR , the authors defined the Age of Data (AoD) for IoT big data processing in MEC networks and proposed a Multi-armed Bandit (MAB) based online learning algorithm to minimize AoD.

In #OTHEREFR , the authors considered a UAV-assisted MEC with NOMA and optimized the trajectory and computation offloading using a successive convex approximation.

In #OTHEREFR , the authors leveraged federated learning (FL) in NOMA-based MEC and used graph theory to improve the communication efficiency of FL and accelerate model convergence.

In #OTHEREFR , the authors studied a data analysis scenario in MEC, where data is generated by energy harvesting technology-powered wireless devices and uploaded to the MEC server for centralized data processing.

They proposed a Lyapunov-based algorithm to schedule resources while satisfying AoI constraints.

However, despite achieving promising performance in various applications, these approaches have neglected to consider the impact of delay, which can be significant when there is a heavy load of requests from multiple users or when downlink resources are limited.

This can result in a degradation of service quality for the users.

Thus, there is a need to discuss "when to serve" these requests, taking into account the incurred delay.

Aoi-Delay Tradeoff In Mobile Edge Caching: A Mixed-Order Drift-Plus-Penalty Algorithm

Abstract: We consider a scheduling problem in a Mobile Edge Caching (MEC) network, where a base station (BS) uploads messages from multiple source nodes (SNs) and transmits them to mobile users (MUs) via downlinks, aiming to jointly optimize the average service Age of Information (AoI) and service delay over MUs. This problem is formulated as a difficult sequential decision making problem with discrete-valued and linearly-constrained design variables. To solve this problem, we first approximate its achievable region by characterizing its superset and subset. The superset is derived based on the rate stability theorem, while the subset is obtained using a novel stochastic policy. We also validate that this subset is substantially identical to the achievable region when the number of schedule resources is large. Additionally, we propose a sufficient condition to check the existence of the solution to the problem. Then, we propose the mixed-order drift-plus-penalty algorithm that uses a dynamic programming (DP) method to optimize the summation over a linear and quadratic Lyapunov drift and a penalty term, to handle the product term over different queue backlogs in the objective function. Finally, by associating the proposed algorithm with the stochastic policy, we demonstrate that it achieves an O(1/V) versus O(V) tradeoff for the average AoI and average delay.

Joint Power And Time Allocation For Noma–Mec Offloading

Abstract: This correspondence considers non-orthogonal multiple access (NOMA) assisted mobile edge computing (MEC), where the power and time allocation is jointly optimized to reduce the energy consumption of computation offloading. Closed-form expressions for the optimal power and time allocation solutions are obtained and used to establish the conditions for determining whether the conventional orthogonal multiple access (OMA), pure NOMA or hybrid NOMA should be used for MEC offloading.

optimal MEC offloading
NOMA-assisted MEC


This improvement is in line with other work using MC Dropout as much as comparisons can be made across domains, including when it was first proposed and tested on a regression task #REFR .

First, we can clearly see that adding Gaussian and Shot noise decimates performance, by a significant degree.

As the model was not trained to be invariant to this type of input noise, this is expected.

Second, our approach generally leads to a slight performance increase over baseline (with one exception on the VIPER dataset with noise), on both clean and noisy data.

Given that acquiring a more reliable measure of uncertainty is the goal, rather than outright performance on the PQ metric, any performance improvement is a welcome addition.

Unfortunately, our approach is not able to compensate for the loss in performance caused by the addition of noise.

Third, we see that while adding more samples does result in a slight performance increase, 5 samples is typically sufficient to generate a reasonable seg-mentation result, especially considering the computational burden of using more and more samples.

Finally, we see that due to the network's tendency to generate many false positives the details around pruning and evaluation are critical.

The per-pixel evaluation, using the same thresholds as the baseline but without the brute-force subsumption of objects, performs worse than baseline, indicating that the brute-force approach is effective with respect to the PQ metric even if not useful from an uncertainty standpoint.

Not applying any thresholds on the other hand harms performance on COCO and VIPER, but actually leads to a slight increase on KITTI-STEP.

Uncertainty Estimation In Deep Learning For Panoptic Segmentation

Abstract: As deep learning-based computer vision algorithms continue to improve and advance the state of the art, their robustness to real-world data continues to lag their performance on datasets. This makes it difficult to bring an algorithm from the lab to the real world. Ensemble-based uncertainty estimation approaches such as Monte Carlo Dropout have been successfully used in many applications in an attempt to address this robustness issue. Unfortunately, it is not always clear if such ensemble-based approaches can be applied to a new problem domain. This is the case with panoptic segmentation, where the structure of the problem and architectures designed to solve it means that unlike image classification or even semantic segmentation, the typical solution of using a mean across samples cannot be directly applied. In this paper, we demonstrate how ensemblebased uncertainty estimation approaches such as Monte Carlo Dropout can be used in the panoptic segmentation domain with no changes to an existing network, providing both improved performance and more importantly a better measure of uncertainty for predictions made by the network. Results are demonstrated quantitatively and qualitatively on the COCO, KITTI-STEP and VIPER datasets.

Dropout As A Bayesian Approximation: Representing Model Uncertainty In Deep Learning

Abstract: Deep learning tools have gained tremendous attention in applied machine learning. However such tools for regression and classification do not capture model uncertainty. In comparison, Bayesian models offer a mathematically grounded framework to reason about model uncertainty, but usually come with a prohibitive computational cost. In this paper we develop a new theoretical framework casting dropout training in deep neural networks (NNs) as approximate Bayesian inference in deep Gaussian processes. A direct result of this theory gives us tools to model uncertainty with dropout NNs -- extracting information from existing models that has been thrown away so far. This mitigates the problem of representing uncertainty in deep learning without sacrificing either computational complexity or test accuracy. We perform an extensive study of the properties of dropout's uncertainty. Various network architectures and non-linearities are assessed on tasks of regression and classification, using MNIST as an example. We show a considerable improvement in predictive log-likelihood and RMSE compared to existing state-of-the-art methods, and finish by using dropout's uncertainty in deep reinforcement learning.


How To Identify The Most Vulnerable Neurons

We address this question by hierarchically clustering #REFR the neurons and visualizing the hierarchy with an interactive dendrogram, as shown in the Neuron Cluster View (Figure 2 -e).

Each row of this weight matrix corresponds to the weight of one class.

respectively; however, for the case where several classes have a non-zero probability, the view shows multiple bars.

The second type of information is the two vulnerability maps: one highlights regions where the image perturbation decreases the benign class's probability the most (Figure 2-d3) , and the other highlights regions where the image perturbation increases the adversarial class's probability the most (Figure 2-d4 ).

While we show the top-20% vulnerable pixels by default, this can be interactively changed with a slider placed in the view.

Once we identify a group of similar neurons, by only examining one representative neuron from the group, we may understand the behaviors of the entire group, making the exploration more efficient.

As a dissimilarity measure used for clustering, we use the 2 distance between two RFs to measure the corresponding neurons' dissimilarity.

We want to note that the IoU between two RFs or the 2 distance between the corresponding activation maps could be used instead; however, based on our comparisons included in our Supplementary Material, there is no obvious advantage of one measure against the others.

When users select a neuron from the Neuron Vulnerability View, its location in the dendrogram is highlighted, as shown in the red line at the far right of Figure 2 -e.

From this location, users can easily identify neurons with similar behavior as similar neurons share the same ancestors and are generally located close to each other in the dendrogram.

Also, to see the RFs summarized across similar neurons, the user can click multiple nodes from the dendrogram, which selects the clicked nodes and their descendants.

Visual Analytics Of Neuron Vulnerability To Adversarial Attacks On Convolutional Neural Networks

Abstract: Adversarial attacks on a convolutional neural network (CNN)-injecting human-imperceptible perturbations into an input image-could fool a high-performance CNN into making incorrect predictions. The success of adversarial attacks raises serious concerns about the robustness of CNNs, and prevents them from being used in safety-critical applications, such as medical diagnosis and autonomous driving. Our work introduces a visual analytics approach to understanding adversarial attacks by answering two questions: (1) which neurons are more vulnerable to attacks and (2) which image features do these vulnerable neurons capture during the prediction? For the first question, we introduce multiple perturbation-based measures to break down the attacking magnitude into individual CNN neurons and rank the neurons by their vulnerability levels. For the second, we identify image features (e.g., cat ears) that highly stimulate a user-selected neuron to augment and validate the neuron's responsibility. Furthermore, we support an interactive exploration of a large number of neurons by aiding with hierarchical clustering based on the neurons' roles in the prediction. To this end, a visual analytics system is designed to incorporate visual reasoning for interpreting adversarial attacks. We validate the effectiveness of our system through multiple case studies as well as feedback from domain experts. CCS Concepts: • Human-centered computing → Visual analytics.

Neuron Cluster View
interactive dendrogram


For the purpose of privacy protection, we leverage Random Fourier Feature (RFF) #REFR to transform the extracted features to reduce the risk of privacy leakage.

To overcome these challenges, we propose HPN, a novel method to achieve pFedHPO.

Instead of a context-free bandit model widely adopted in existing FedHPO works (e.g., FedEx #OTHEREFR ), we design a policy network that takes a client encoding as input and outputs a distribution over the original search space.

Therefore, each client has its specific encoding to determine its hyperparameters personally.

Meanwhile, this policy network's parameters are updated based on the observations collected from all the clients, which can be regarded as improving the sample efficiency via model sharing.

In HPN, a client encoding is calculated based on the training sample of that client to reflect the similarity among clients.

Furthermore, we design a mechanism to conduct low-fidelity evaluations of hyperparameter configurations, which reduces variance for the signals used to update our policy and, in the meantime, alleviates the impact of model parameters' state.

Finally, we conduct extensive experiments on FedHPO tasks of various domains, where the generalization error of the hyperparameter configurations searched by HPN is lower than that of related baselines.

Our contributions are summarized as follows:

• We are the first to systematically explore pFedHPO, discussing its setting, challenges, and critical problemsolving factors.

• We propose a novel pFedHPO method HPN, which satisfies the sample efficiency and privacy preservation requirements.

Hpn: Personalized Federated Hyperparameter Optimization

Abstract: Numerous research studies in the field of federated learning (FL) have attempted to use personalization to address the heterogeneity among clients, one of FL's most crucial and challenging problems. However, existing works predominantly focus on tailoring models. Yet, due to the heterogeneity of clients, they may each require different choices of hyperparameters, which have not been studied so far. We pinpoint two challenges of personalized federated hyperparameter optimization (pFedHPO): handling the exponentially increased search space and characterizing each client without compromising its data privacy. To overcome them, we propose learning a HyperParameter Network (HPN) fed with client encoding to decide personalized hyperparameters. The client encoding is calculated with a random projectionbased procedure to protect each client's privacy. Besides, we design a novel mechanism to debias the low-fidelity function evaluation samples for learning HPN. We conduct extensive experiments on FL tasks from various domains, demonstrating the superiority of HPN.

Random Fourier Features For Kernel Ridge Regression: Approximation Bounds And Statistical Guarantees

Abstract: Random Fourier features is one of the most popular techniques for scaling up kernel methods, such as kernel ridge regression. However, despite impressive empirical results, the statistical properties of random Fourier features are still not well understood. In this paper we take steps toward filling this gap. Specifically, we approach random Fourier features from a spectral matrix approximation point of view, give tight bounds on the number of Fourier features required to achieve a spectral approximation, and show how spectral matrix approximation bounds imply statistical guarantees for kernel ridge regression. Qualitatively, our results are twofold: on the one hand, we show that random Fourier feature approximation can provably speed up kernel ridge regression under reasonable assumptions. At the same time, we show that the method is suboptimal, and sampling from a modified distribution in Fourier space, given by the leverage function of the kernel, yields provably better performance. We study this optimal sampling distribution for the Gaussian kernel, achieving a nearly complete characterization for the case of low-dimensional bounded datasets. Based on this characterization, we propose an efficient sampling scheme with guarantees superior to random Fourier features in this regime.


Following this idea, the Automated Audio Captioning (AAC) task appeared in 2017 #REFR and aims to create systems that generate a sentence written in natural language that describes an audio file.

In recent years, new machine learning systems have been significantly improved for text processing, generation, and understanding, leading to the use of natural language as a global interface between humans and machines.

Free-form text can contain much more information than a predefined set of classes, which could improve the machine understanding of our world.

In audio, most of the tasks are focused on classification and localization of sound events.

The audio can contain various sound events (human, natural, domestic, urban, music, effects...) of different lengths, recorded with different devices and in different scenes.

The description can contain any kind of detail in the audio, with temporal or spatial relations between them (followed by, in the background...) or different characterizations (high-pitched, short, repetitive...).

Since the descriptions are written by humans, we need to consider different words used to describe similar sounds (Birds are calling / chirping / singing / tweeting), different sentence structures (A door that needs to be oiled / A door with squeaky hinges), subjectivity (Man speaks in a foreign language), high-level descriptions (A vulgar man speaks / Unintelligible conversation), and vagueness (Someone speaks instead of A man gives a speech over a reverberating microphone).

In AAC, most approaches use deep learning models trained with the standard Cross-Entropy (CE) loss.

However, this loss tends to generate repetitive and generic content #OTHEREFR and does not take into account synonyms, various sentences structures or the semantic closeness.

Multitask Learning In Audio Captioning: A Sentence Embedding Regression Loss Acts As A Regularizer

Abstract: In this work, we propose to study the performance of a model trained with a sentence embedding regression loss component for the Automated Audio Captioning task. This task aims to build systems that can describe audio content with a single sentence written in natural language. Most systems are trained with the standard Cross-Entropy loss, which does not take into account the semantic closeness of the sentence. We found that adding a sentence embedding loss term reduces overfitting, but also increased SPIDEr from 0.397 to 0.418 in our first setting on the AudioCaps corpus. When we increased the weight decay value, we found our model to be much closer to the current stateof-the-art methods, with a SPIDEr score up to 0.444 compared to a 0.475 score. Moreover, this model uses eight times less trainable parameters. In this training setting, the sentence embedding loss has no more impact on the model performance.

Automated Audio Captioning With Recurrent Neural Networks

Abstract: We present the first approach to automated audio captioning. We employ an encoder-decoder scheme with an alignment model in between. The input to the encoder is a sequence of log mel-band energies calculated from an audio file, while the output is a sequence of words, i.e. a caption. The encoder is a multi-layered, bi-directional gated recurrent unit (GRU) and the decoder a multi-layered GRU with a classification layer connected to the last GRU of the decoder. The classification layer and the alignment model are fully connected layers with shared weights between timesteps. The proposed method is evaluated using data drawn from a commercial sound effects library, ProSound Effects. The resulting captions were rated through metrics utilized in machine translation and image captioning fields. Results from metrics show that the proposed method can predict words appearing in the original caption, but not always correctly ordered.


As presented in the main paper, the original dataset is pre-processed to 5000-dimension bag-of words features following #REFR .

And the target shift data is created by randomly dropping 50% negative reviews.

Model Structure Representation learner: [5000, 1000, 500, 100] MLP net using 0.7 dropout rate with Relu activation added after each hidden layer and finally output a 100-dimension feature representation.

Predictor and duplicate predictor: [100, 2] linear transformation followed by a log softmax layer transforming the 100dimension feature to 2-class log probabilities.

Loss function: we choose the "negative log-likelihood loss" as the loss function.

Computing Resources The experiments were run on a server with 6 CPUs and 1 GPU of 32GB memory.

Algorithm-Dependent Bounds For Representation Learning Of Multi-Source Domain Adaptation

Abstract: We use information-theoretic tools to derive a novel analysis of Multi-source Domain Adaptation (MDA) from the representation learning perspective. Concretely, we study joint distribution alignment for supervised MDA with few target labels and unsupervised MDA with pseudo labels, where the latter is relatively hard and less commonly studied. We further provide algorithm-dependent generalization bounds for these two settings, where the generalization is characterized by the mutual information between the parameters and the data. Then we propose a novel deep MDA algorithm, implicitly addressing the target shift through joint alignment. Finally, the mutual information bounds are extended to this algorithm providing a nonvacuous gradient-norm estimation. The proposed algorithm has comparable performance to the state-of-the-art on target-shifted MDA benchmark with improved memory efficiency. 1 We use the terminology of target shift in the rest of the paper to avoid confusion with the label shift assumption, where S(X|Y) = T (X|Y), S(Y) = T (Y).

Marginalized Denoising Autoencoders For Domain Adaptation

Abstract: Stacked denoising autoencoders (SDAs) have been successfully used to learn new representations for domain adaptation. Recently, they have attained record accuracy on standard benchmark tasks of sentiment analysis across different text domains. SDAs learn robust data representations by reconstruction, recovering original features from data that are artificially corrupted with noise. In this paper, we propose marginalized SDA (mSDA) that addresses two crucial limitations of SDAs: high computational cost and lack of scalability to high-dimensional features. In contrast to SDAs, our approach of mSDA marginalizes noise and thus does not require stochastic gradient descent or other optimization algorithms to learn parameters--in fact, they are computed in closed-form. Consequently, mSDA, which can be implemented in only 20 lines of MATLAB™, significantly speeds up SDAs by two orders of magnitude. Furthermore, the representations learnt by mSDA are as effective as the traditional SDAs, attaining almost identical accuracies in benchmark tasks.


Traditional sparse representation approaches can be considered as unsupervised methods that either ignore label information #REFR or learn prototypes for each label separately.

The prototype-based approaches cluster samples and then choose one or a few samples or their representatives in each cluster #OTHEREFR .

Dimensionality-reductionbased approaches, such as product quantization #OTHEREFR and hashing #OTHEREFR , focus on encoding high-dimensional feature spaces densely to achieve speed-up in searchbased methods as well as reducing the memory costs.

Our proposed approach belongs to the third group of scalable methods, transform-based approaches #OTHEREFR , that treat image annotation as a multi-label problem.

In these approaches, both visual and semantic modalities are incorporated into the learning procedure for transforming input data into another space with higher levels of discrimination.

One of the successful techniques in this category is sparse representation whose objective is to represent each pattern just using the linear combination of a few numbers of prototypes.

In recent years, many researchers have focused on embedding label information into the prototype learning procedure, generally known as discriminative #OTHEREFR or coupled #OTHEREFR dictionary learning, extensively applied for multi-label classification problems #OTHEREFR .

Discriminative sparse models have many applications in image classification, super-resolution #OTHEREFR , fault-diagnosis, etc.

class-specific and shared discriminative dictionary learning (CASDDL) method #OTHEREFR aims to classify the steel sheets based on the Fisher discrimination method.

They strive to extract the discriminative features for each class separately (inter-class information), along with a shared sub-dictionary which is common between all the classes for extracting the intra-class information. Li et al.

#OTHEREFR offered a weighted regularization approach to tackle the noisy images.

Toward Real-Time Image Annotation Using Marginalized Coupled Dictionary Learning

Abstract: In most image retrieval systems, images include various high-level semantics, called tags or annotations. Virtually all the state-of-the-art image annotation methods that handle imbalanced labeling are search-based techniques which are time-consuming. In this paper, a novel coupled dictionary learning approach is proposed to learn a limited number of visual prototypes and their corresponding semantics simultaneously. This approach leads to a real-time image annotation procedure. Another contribution of this paper is that utilizes a marginalized loss function instead of the squared loss function that is inappropriate for image annotation with imbalanced labels. We have employed a marginalized loss function in our method to leverage a simple and effective method of prototype updating. Meanwhile, we have introduced 1 regularization on semantic prototypes to preserve the sparse and imbalanced nature of labels in learned semantic prototypes. Finally, comprehensive experimental results on various datasets demonstrate the efficiency of the pro-Authors contributed equally on this research.

Online Learning For Matrix Factorization And Sparse Coding

Abstract: Sparse coding--that is, modelling data vectors as sparse linear combinations of basis elements--is widely used in machine learning, neuroscience, signal processing, and statistics. This paper focuses on the large-scale matrix factorization problem that consists of learning the basis set in order to adapt it to specific data. Variations of this problem include dictionary learning in signal processing, non-negative matrix factorization and sparse principal component analysis. In this paper, we propose to address these tasks with a new online optimization algorithm, based on stochastic approximations, which scales up gracefully to large data sets with millions of training samples, and extends naturally to various matrix factorization formulations, making it suitable for a wide range of learning problems. A proof of convergence is presented, along with experiments with natural images and genomic data demonstrating that it leads to state-of-the-art performance in terms of speed and optimization for both small and large data sets.


We use Scikit-learn #REFR , one of the popular and widely used tools to implement the techniques.

According to our comparative study, the relationship between question classes and their corresponding feature values might be complex.

We thus choose five popular supervised machine learning techniques with different learning strategies.

They are -i) Decision Tree #OTHEREFR (DT) ii) Random Forest (RF) #OTHEREFR , iii) Artificial Neural Network (ANN) #OTHEREFR , iv) K-Nearest Neighbors (KNN) #OTHEREFR , and v) Gaussian Naive Bayes (GNB) #OTHEREFR .

In particular, we choose these machine learning algorithms because they are widely used in the relevant studies #OTHEREFR .

We thus believe they can build reliable models to classify promoted and discoursed questions.

Tuning parameters in classifiers is important because it changes the heuristics determining how they learn #OTHEREFR .

For example, it controls the number of decision trees to use in RF or the number of clusters in KNN.

Models trained with suboptimal parameter settings may underperform as parameter settings depend on the dataset #OTHEREFR .

To select the best model configuration, we use GridSearchCV, the cross-validated grid search algorithm of Scikit-learn #OTHEREFR .

Do Subjectivity And Objectivity Always Agree? A Case Study With Stack Overflow Questions

Abstract: In Stack Overflow (SO), the quality of posts (i.e., questions and answers) is subjectively evaluated by users through a voting mechanism. The net votes (upvotes − downvotes) obtained by a post are often considered an approximation of its quality. However, about half of the questions that received working solutions got more downvotes than upvotes. Furthermore, about 18% of the accepted answers (i.e., verified solutions) also do not score the maximum votes. All these counter-intuitive findings cast doubts on the reliability of the evaluation mechanism employed at SO. Moreover, many users raise concerns against the evaluation, especially downvotes to their posts. Therefore, rigorous verification of the subjective evaluation is highly warranted to ensure a non-biased and reliable quality assessment mechanism. In this paper, we compare the subjective assessment of questions with their objective assessment using 2.5 million questions and ten text analysis metrics. According to our investigation, four objective metrics agree with the subjective evaluation, two do not agree, one either agrees or disagrees, and the remaining three neither agree nor disagree with the subjective evaluation. We then develop machine learning models to classify the promoted and discouraged questions. Our models outperform the state-ofthe-art models with a maximum of about 76%-87% accuracy.

Scikit-Learn: Machine Learning In Python

Abstract: Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. This package focuses on bringing machine learning to non-specialists using a general-purpose high-level language. Emphasis is put on ease of use, performance, documentation, and API consistency. It has minimal dependencies and is distributed under the simplified BSD license, encouraging its use in both academic and commercial settings. Source code, binaries, and documentation can be downloaded from http://scikit-learn.sourceforge.net.