id,title,abstract
2128438887,An index to quantify an individual's scientific research output,"I propose the index h, defined as the number of papers with citation number > or =h, as a useful index to characterize the scientific output of a researcher."
2150220236,"Software survey: VOSviewer, a computer program for bibliometric mapping","We present VOSviewer, a freely available computer program that we have developed for constructing and viewing bibliometric maps. Unlike most computer programs that are used for bibliometric mapping, VOSviewer pays special attention to the graphical representation of bibliometric maps. The functionality of VOSviewer is especially useful for displaying large bibliometric maps in an easy-to-interpret way. The paper consists of three parts. In the first part, an overview of VOSviewer's functionality for displaying bibliometric maps is provided. In the second part, the technical implementation of specific parts of the program is discussed. Finally, in the third part, VOSviewer's ability to handle large maps is demonstrated by using the program to construct and display a co-citation map of 5,000 major scientific journals."
1965631677,The Increasing Dominance of Teams in Production of Knowledge,"We have used 19.9 million papers over 5 decades and 2.1 million patents to demonstrate that teams increasingly dominate solo authors in the production of knowledge. Research is increasingly done in teams across nearly all fields. Teams typically produce more frequently cited research than individuals do, and this advantage has been increasing over time. Teams now also produce the exceptionally high-impact research, even where that distinction was once the domain of solo authors. These results are detailed for sciences and engineering, social sciences, arts and humanities, and patents, suggesting that the process of knowledge creation has fundamentally changed."
2019753053,What do citation counts measure? A review of studies on citing behavior,"Purpose The purpose of this paper is to present a narrative review of studies on the citing behavior of scientists, covering mainly research published in the last 15 years. Based on the results of these studies, the paper seeks to answer the question of the extent to which scientists are motivated to cite a publication not only to acknowledge intellectual and cognitive influences of scientific peers, but also for other, possibly non‐scientific, reasons. Design/methodology/approach The review covers research published from the early 1960s up to mid‐2005 (approximately 30 studies on citing behavior‐reporting results in about 40 publications). Findings The general tendency of the results of the empirical studies makes it clear that citing behavior is not motivated solely by the wish to acknowledge intellectual and cognitive influences of colleague scientists, since the individual studies reveal also other, in part non‐scientific, factors that play a part in the decision to cite. However, the results of the studies must also be deemed scarcely reliable: the studies vary widely in design, and their results can hardly be replicated. Many of the studies have methodological weaknesses. Furthermore, there is evidence that the different motivations of citers are “not so different or ‘randomly given’ to such an extent that the phenomenon of citation would lose its role as a reliable measure of impact”. Originality/value Given the increasing importance of evaluative bibliometrics in the world of scholarship, the question “What do citation counts measure?” is a particularly relevant and topical issue."
2122831750,The Impact of Research Collaboration on Scientific Productivity,"Based on the curricula vitae and survey responses of 443 academic scientists affiliated with university research centers in the USA, we examine the longstanding assumption that research collaboration has a positive effect on publishing productivity. Since characteristics of the individual and the work environment are endogenously related to both collaboration and productivity, this study focuses on the mediating effect of collaboration on publishing productivity. By using the two-stage least squares analysis, the findings indicate that in the presence of moderating variables such as age, rank, grant, gender, marital status, family relations, citizenship, job satisfaction, perceived discrimination, and collaboration strategy, the simple number (‘normal count’) of peer-reviewed journal papers is strongly and significantly associated with the number of collaborators. However, the net impacts of collaboration are less clear. When we apply the same model and examine productivity by ‘fractional count’, dividing the number of publications by the number of authors, we find that number of collaborators is not a significant predictor of publishing productivity. In both cases, ‘normal count’ and ‘fractional count’, we find significant effects of research grants, citizenship, collaboration strategy, and scientific field. We believe that it is important to understand the effects of the individual and environmental factors for developing effective strategies to exploit the potential benefits of collaboration. We note that our focus is entirely at the individual level, and some of the most important benefits of collaboration may accrue to groups, institutions, and scientific fields."
2120109270,"Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses","The evolution of the electronic age has led to the development of numerous medical databases on the World Wide Web, offering search facilities on a particular subject and the ability to perform citation analysis. We compared the content coverage and practical utility of PubMed, Scopus, Web of Science, and Google Scholar. The official Web pages of the databases were used to extract information on the range of journals covered, search facilities and restrictions, and update frequency. We used the example of a keyword search to evaluate the usefulness of these databases in biomedical information retrieval and a specific published article to evaluate their utility in performing citation analysis. All databases were practical in use and offered numerous search facilities. PubMed and Google Scholar are accessed for free. The keyword search with PubMed offers optimal update frequency and includes online early articles; other databases can rate articles by number of citations, as an index of importance. For citation analysis, Scopus offers about 20% more coverage than Web of Science, whereas Google Scholar offers results of inconsistent accuracy. PubMed remains an optimal tool in biomedical electronic research. Scopus covers a wider journal range, of help both in keyword searching and citation analysis, but it is currently limited to recent articles (published after 1995) compared with Web of Science. Google Scholar, as for the Web in general, can help in the retrieval of even the most obscure information but its use is marred by inadequate, less often updated, citation information."
2097777089,Coauthorship networks and patterns of scientific collaboration,"By using data from three bibliographic databases in biology, physics, and mathematics, respectively, networks are constructed in which the nodes are scientists, and two scientists are connected if they have coauthored a paper. We use these networks to answer a broad variety of questions about collaboration patterns, such as the numbers of papers authors write, how many people they write them with, what the typical distance between scientists is through the network, and how patterns of collaboration vary between subjects and over time. We also summarize a number of recent results by other authors on coauthorship patterns."
4238591974,CiteSpace II: Detecting and visualizing emerging trends and transient patterns in scientific literature,"Abstract This article describes the latest development of a generic approach to detecting and visualizing emerging trends and transient patterns in scientific literature. The work makes substantial theoretical and methodological contributions to progressive knowledge domain visualization. A specialty is conceptualized and visualized as a time‐variant duality between two fundamental concepts in information science: research fronts and intellectual bases. A research front is defined as an emergent and transient grouping of concepts and underlying research issues. The intellectual base of a research front is its citation and co‐citation footprint in scientific literature—an evolving network of scientific publications cited by research‐front concepts. Kleinberg's (2002) burst‐detection algorithm is adapted to identify emergent research‐front concepts. Freeman's (1979) betweenness centrality metric is used to highlight potential pivotal points of paradigm shift over time. Two complementary visualization views are designed and implemented: cluster views and time‐zone views. The contributions of the approach are that (a) the nature of an intellectual base is algorithmically and temporally identified by emergent research‐front terms, (b) the value of a co‐citation cluster is explicitly interpreted in terms of research‐front concepts, and (c) visually prominent and algorithmically detected pivotal points substantially reduce the complexity of a visualized network. The modeling and visualization process is implemented in CiteSpace II, a Java application, and applied to the analysis of two research fields: mass extinction (1981–2004) and terrorism (1990–2003). Prominent trends and pivotal points in visualized networks were verified in collaboration with domain experts, who are the authors of pivotal‐point articles. Practical implications of the work are discussed. A number of challenges and opportunities for future studies are identified."
1982218215,Does the h index have predictive power?,"Bibliometric measures of individual scientific achievement are of particular interest if they can be used to predict future achievement. Here we report results of an empirical study of the predictive power of the h index compared with other indicators. Our findings indicate that the h index is better than other indicators considered (total citation count, citations per paper, and total paper count) in predicting future scientific achievement. We discuss reasons for the superiority of the h index."
2073023209,Do Altmetrics Work? Twitter and Ten Other Social Web Services,"Altmetric measurements derived from the social web are increasingly advocated and used as early indicators of article impact and usefulness. Nevertheless, there is a lack of systematic scientific evidence that altmetrics are valid proxies of either impact or utility although a few case studies have reported medium correlations between specific altmetrics and citation rates for individual journals or fields. To fill this gap, this study compares 11 altmetrics with Web of Science citations for 76 to 208,739 PubMed articles with at least one altmetric mention in each case and up to 1,891 journals per metric. It also introduces a simple sign test to overcome biases caused by different citation and usage windows. Statistically significant associations were found between higher metric scores and higher citations for articles with positive altmetric scores in all cases with sufficient evidence (Twitter, Facebook wall posts, research highlights, blogs, mainstream media and forums) except perhaps for Google+ posts. Evidence was insufficient for LinkedIn, Pinterest, question and answer sites, and Reddit, and no conclusions should be drawn about articles with zero altmetric scores or the strength of any correlation between altmetrics and citations. Nevertheless, comparisons between citations and metric values for articles published at different times, even within the same year, can remove or reverse this association and so publishers and scientometricians should consider the effect of time when using altmetrics to rank articles. Finally, the coverage of all the altmetrics except for Twitter seems to be low and so it is not clear if they are prevalent enough to be useful in practice."
2151866568,Universality of citation distributions: Toward an objective measure of scientific impact,"We study the distributions of citations received by a single publication within several disciplines, spanning broad areas of science. We show that the probability that an article is cited $c$ times has large variations between different disciplines, but all distributions are rescaled on a universal curve when the relative indicator $c_f=c/c_0$ is considered, where $c_0$ is the average number of citations per article for the discipline. In addition we show that the same universal behavior occurs when citation distributions of articles published in the same field, but in different years, are compared. These findings provide a strong validation of $c_f$ as an unbiased indicator for citation performance across disciplines and years. Based on this indicator, we introduce a generalization of the h-index suitable for comparing scientists working in different fields."
2128186353,Who's Afraid of Peer Review?,A spoof paper concocted by Science reveals little or no scrutiny at many open-access journals.
1965964177,Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact,"Citations in peer-reviewed articles and the impact factor are generally accepted measures of scientific impact. Web 2.0 tools such as Twitter, blogs or social bookmarking tools provide the possibility to construct innovative article-level or journal-level metrics to gauge impact and influence. However, the relationship of the these new metrics to traditional metrics such as citations is not known.(1) To explore the feasibility of measuring social impact of and public attention to scholarly articles by analyzing buzz in social media, (2) to explore the dynamics, content, and timing of tweets relative to the publication of a scholarly article, and (3) to explore whether these metrics are sensitive and specific enough to predict highly cited articles.Between July 2008 and November 2011, all tweets containing links to articles in the Journal of Medical Internet Research (JMIR) were mined. For a subset of 1573 tweets about 55 articles published between issues 3/2009 and 2/2010, different metrics of social media impact were calculated and compared against subsequent citation data from Scopus and Google Scholar 17 to 29 months later. A heuristic to predict the top-cited articles in each issue through tweet metrics was validated.A total of 4208 tweets cited 286 distinct JMIR articles. The distribution of tweets over the first 30 days after article publication followed a power law (Zipf, Bradford, or Pareto distribution), with most tweets sent on the day when an article was published (1458/3318, 43.94% of all tweets in a 60-day period) or on the following day (528/3318, 15.9%), followed by a rapid decay. The Pearson correlations between tweetations and citations were moderate and statistically significant, with correlation coefficients ranging from .42 to .72 for the log-transformed Google Scholar citations, but were less clear for Scopus citations and rank correlations. A linear multivariate model with time and tweets as significant predictors (P < .001) could explain 27% of the variation of citations. Highly tweeted articles were 11 times more likely to be highly cited than less-tweeted articles (9/12 or 75% of highly tweeted article were highly cited, while only 3/43 or 7% of less-tweeted articles were highly cited; rate ratio 0.75/0.07 = 10.75, 95% confidence interval, 3.4-33.6). Top-cited articles can be predicted from top-tweeted articles with 93% specificity and 75% sensitivity.Tweets can predict highly cited articles within the first 3 days of article publication. Social media activity either increases citations or reflects the underlying qualities of the article that also predict citations, but the true use of these metrics is to measure the distinct concept of social impact. Social impact measures based on tweets are proposed to complement traditional citation metrics. The proposed twimpact factor may be a useful and timely metric to measure uptake of research findings and to filter research findings resonating with the public in real time."
2089604928,Citation Advantage of Open Access Articles,"Open access (OA) to the research literature has the potential to accelerate recognition and dissemination of research findings, but its actual effects are controversial. This was a longitudinal bibliometric analysis of a cohort of OA and non-OA articles published between June 8, 2004, and December 20, 2004, in the same journal (PNAS: Proceedings of the National Academy of Sciences). Article characteristics were extracted, and citation data were compared between the two groups at three different points in time: at ""quasi-baseline"" (December 2004, 0-6 mo after publication), in April 2005 (4-10 mo after publication), and in October 2005 (10-16 mo after publication). Potentially confounding variables, including number of authors, authors' lifetime publication count and impact, submission track, country of corresponding author, funding organization, and discipline, were adjusted for in logistic and linear multiple regression models. A total of 1,492 original research articles were analyzed: 212 (14.2% of all articles) were OA articles paid by the author, and 1,280 (85.8%) were non-OA articles. In April 2005 (mean 206 d after publication), 627 (49.0%) of the non-OA articles versus 78 (36.8%) of the OA articles were not cited (relative risk = 1.3 [95% Confidence Interval: 1.1-1.6]; p = 0.001). 6 mo later (mean 288 d after publication), non-OA articles were still more likely to be uncited (non-OA: 172 [13.6%], OA: 11 [5.2%]; relative risk = 2.6 [1.4-4.7]; p < 0.001). The average number of citations of OA articles was higher compared to non-OA articles (April 2005: 1.5 [SD = 2.5] versus 1.2 [SD = 2.0]; Z = 3.123; p = 0.002; October 2005: 6.4 [SD = 10.4] versus 4.5 [SD = 4.9]; Z = 4.058; p < 0.001). In a logistic regression model, controlling for potential confounders, OA articles compared to non-OA articles remained twice as likely to be cited (odds ratio = 2.1 [1.5-2.9]) in the first 4-10 mo after publication (April 2005), with the odds ratio increasing to 2.9 (1.5-5.5) 10-16 mo after publication (October 2005). Articles published as an immediate OA article on the journal site have higher impact than self-archived or otherwise openly accessible OA articles. We found strong evidence that, even in a journal that is widely available in research libraries, OA articles are more immediately recognized and cited by peers than non-OA articles published in the same journal. OA is likely to benefit science by accelerating dissemination and uptake of research findings."
2005950835,Bias in peer review,"Research on bias in peer review examines scholarly communication and funding processes to assess the epistemic and social legitimacy of the mechanisms by which knowledge communities vet and self‐regulate their work. Despite vocal concerns, a closer look at the empirical and methodological limitations of research on bias raises questions about the existence and extent of many hypothesized forms of bias. In addition, the notion of bias is predicated on an implicit ideal that, once articulated, raises questions about the normative implications of research on bias in peer review. This review provides a brief description of the function, history, and scope of peer review; articulates and critiques the conception of bias unifying research on bias in peer review; characterizes and examines the empirical, methodological, and normative claims of bias in peer review research; and assesses possible alternatives to the status quo. We close by identifying ways to expand conceptions and studies of bias to contend with the complexity of social interactions among actors involved directly and indirectly in peer review."
2164277894,Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective,"An extensive analysis of the presence of different altmetric indicators provided by Altmetric.com across scientific fields is presented, particularly focusing on their relationship with citations. Our results confirm that the presence and density of social media altmetric counts are still very low and not very frequent among scientific publications, with 15%–24% of the publications presenting some altmetric activity and concentrated on the most recent publications, although their presence is increasing over time. Publications from the social sciences, humanities, and the medical and life sciences show the highest presence of altmetrics, indicating their potential value and interest for these fields. The analysis of the relationships between altmetrics and citations confirms previous claims of positive correlations but is relatively weak, thus supporting the idea that altmetrics do not reflect the same kind of impact as citations. Also, altmetric counts do not always present a better filtering of highly‐cited publications than journal citation scores. Altmetric scores (particularly mentions in blogs) are able to identify highly‐cited publications with higher levels of precision than journal citation scores ( JCS ), but they have a lower level of recall. The value of altmetrics as a complementary tool of citation analysis is highlighted, although more research is suggested to disentangle the potential meaning and value of altmetric indicators for research evaluation."
4255356361,Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar,"Abstract The Institute for Scientific Information's (ISI, now Thomson Scientific, Philadelphia, PA) citation databases have been used for decades as a starting point and often as the only tools for locating citations and/or conducting citation analyses. The ISI databases (or Web of Science [WoS]), however, may no longer be sufficient because new databases and tools that allow citation searching are now available. Using citations to the work of 25 library and information science (LIS) faculty members as a case study, the authors examine the effects of using Scopus and Google Scholar (GS) on the citation counts and rankings of scholars as measured by WoS. Overall, more than 10,000 citing and purportedly citing documents were examined. Results show that Scopus significantly alters the relative ranking of those scholars that appear in the middle of the rankings and that GS stands out in its coverage of conference proceedings as well as international, non‐English language journals. The use of Scopus and GS, in addition to WoS, helps reveal a more accurate and comprehensive picture of the scholarly impact of authors. The WoS data took about 100 hours of collecting and processing time, Scopus consumed 200 hours, and GS a grueling 3,000 hours."
2040220320,Institutional Repositories: Essential Infrastructure For Scholarship In The Digital Age,"In the fall of 2002, something extraordinary occurred in the continuing networked information revolution, shifting the dynamic among individually driven innovation, institutional progress, and the evolution of disciplinary scholarly practices. The development of institutional repositories emerged as a new strategy that allows universities to apply serious, systematic leverage to accelerate changes taking place in scholarship and scholarly communication, both moving beyond their historic relatively passive role of supporting established publishers in modernizing scholarly publishing through the licensing of digital content, and also scaling up beyond ad-hoc alliances, partnerships, and support arrangements with a few select faculty pioneers exploring more transformative new uses of the digital medium."
2156487032,‘Predatory’ open access: a longitudinal study of article volumes and market characteristics,"A negative consequence of the rapid growth of scholarly open access publishing funded by article processing charges is the emergence of publishers and journals with highly questionable marketing and peer review practices. These so-called predatory publishers are causing unfounded negative publicity for open access publishing in general. Reports about this branch of e-business have so far mainly concentrated on exposing lacking peer review and scandals involving publishers and journals. There is a lack of comprehensive studies about several aspects of this phenomenon, including extent and regional distribution. After an initial scan of all predatory publishers and journals included in the so-called Beall's list, a sample of 613 journals was constructed using a stratified sampling method from the total of over 11,000 journals identified. Information about the subject field, country of publisher, article processing charge and article volumes published between 2010 and 2014 were manually collected from the journal websites. For a subset of journals, individual articles were sampled in order to study the country affiliation of authors and the publication delays. Over the studied period, predatory journals have rapidly increased their publication volumes from 53,000 in 2010 to an estimated 420,000 articles in 2014, published by around 8,000 active journals. Early on, publishers with more than 100 journals dominated the market, but since 2012 publishers in the 10–99 journal size category have captured the largest market share. The regional distribution of both the publisher's country and authorship is highly skewed, in particular Asia and Africa contributed three quarters of authors. Authors paid an average article processing charge of 178 USD per article for articles typically published within 2 to 3 months of submission. Despite a total number of journals and publishing volumes comparable to respectable (indexed by the Directory of Open Access Journals) open access journals, the problem of predatory open access seems highly contained to just a few countries, where the academic evaluation practices strongly favor international publication, but without further quality checks."
2741809807,The state of OA: a large-scale analysis of the prevalence and impact of Open Access articles,"Despite growing interest in Open Access (OA) to scholarly literature, there is an unmet need for large-scale, up-to-date, and reproducible studies assessing the prevalence and characteristics of OA. We address this need using oaDOI, an open online service that determines OA status for 67 million articles. We use three samples, each of 100,000 articles, to investigate OA in three populations: (1) all journal articles assigned a Crossref DOI, (2) recent journal articles indexed in Web of Science, and (3) articles viewed by users of Unpaywall, an open-source browser extension that lets users find OA articles using oaDOI. We estimate that at least 28% of the scholarly literature is OA (19M in total) and that this proportion is growing, driven particularly by growth in Gold and Hybrid. The most recent year analyzed (2015) also has the highest percentage of OA (45%). Because of this growth, and the fact that readers disproportionately access newer articles, we find that Unpaywall users encounter OA quite frequently: 47% of articles they view are OA. Notably, the most common mechanism for OA is not Gold, Green, or Hybrid OA, but rather an under-discussed category we dub Bronze: articles made free-to-read on the publisher website, without an explicit Open license. We also examine the citation impact of OA articles, corroborating the so-called open-access citation advantage: accounting for age and discipline, OA articles receive 18% more citations than average, an effect driven primarily by Green and Hybrid OA. We encourage further research using the free oaDOI service, as a way to inform OA policy and practice."
2086496065,Do Open-Access Articles Have a Greater Research Impact?,"Although many authors believe that their work has a greater research impact if it is freely available, studies to demonstrate that impact are few. This study looks at articles in four disciplines at varying stages of adoption of open access—philosophy, political science, electrical and electronic engineering and mathematics—to see whether they have a greater impact as measured by citations in the ISI Web of Science database when their authors make them freely available on the Internet. The finding is that, across all four disciplines, freely available articles do have a greater research impact. Shedding light on this category of open access reveals that scholars in diverse disciplines are adopting open-access practices and being rewarded for it."
2793071066,Science of science,"Identifying fundamental drivers of science and developing predictive models to capture its evolution are instrumental for the design of policies that can improve the scientific enterprise-for example, through enhanced career paths for scientists, better performance evaluation for organizations hosting research, discovery of novel effective funding vehicles, and even identification of promising regions along the scientific frontier. The science of science uses large-scale data on the production of science to search for universal and domain-specific patterns. Here, we review recent developments in this transdisciplinary field."
2108680868,"Science mapping software tools: Review, analysis, and cooperative study among tools","Science mapping aims to build bibliometric maps that describe how specific disciplines, scientific domains, or research fields are conceptually, intellectually, and socially structured. Different techniques and software tools have been proposed to carry out science mapping analysis. The aim of this article is to review, analyze, and compare some of these software tools, taking into account aspects such as the bibliometric techniques available and the different kinds of analysis."
1655306493,The Oligopoly of Academic Publishers in the Digital Era,"The consolidation of the scientific publishing industry has been the topic of much debate within and outside the scientific community, especially in relation to major publishers' high profit margins. However, the share of scientific output published in the journals of these major publishers, as well as its evolution over time and across various disciplines, has not yet been analyzed. This paper provides such analysis, based on 45 million documents indexed in the Web of Science over the period 1973-2013. It shows that in both natural and medical sciences (NMS) and social sciences and humanities (SSH), Reed-Elsevier, Wiley-Blackwell, Springer, and Taylor & Francis increased their share of the published output, especially since the advent of the digital era (mid-1990s). Combined, the top five most prolific publishers account for more than 50% of all papers published in 2013. Disciplines of the social sciences have the highest level of concentration (70% of papers from the top five publishers), while the humanities have remained relatively independent (20% from top five publishers). NMS disciplines are in between, mainly because of the strength of their scientific societies, such as the ACS in chemistry or APS in physics. The paper also examines the migration of journals between small and big publishing houses and explores the effect of publisher change on citation impact. It concludes with a discussion on the economics of scholarly publishing."
2124689612,Quantifying Long-Term Scientific Impact,"Citation Grabbers Is there quantifiable regularity and predictability in citation patterns? It is clear that papers that have been cited frequently tend to accumulate more citations. It is also clear that, with time, even the most novel paper loses its currency. Some papers, however, seem to have an inherent “fitness” that can be interpreted as a community's response to the research. Wang et al. (p. 127 ; see the Perspective by Evans ) developed a mechanistic model to predict citation history. The model links a paper's ultimate impact, represented by the total number of citations the paper will ever receive, to a single measurable parameter inferred from its early citation history. The model was used to identify factors that influence a journal's impact factor."
2089644168,The Structure of a Social Science Collaboration Network: Disciplinary Cohesion from 1963 to 1999,"Has sociology become more socially integrated over the last 30 years? Recent work in the sociology of knowledge demonstrates a direct linkage between social interaction patterns and the structure of ideas, suggesting that scientific collaboration networks affect scientific practice. I test three competing models for sociological collaboration networks and find that a structurally cohesive core that has been growing steadily since the early 1960s characterizes the discipline's coauthorship network. The results show that participation in the sociology collaboration network depends on research specialty and that quantitative work is more likely to be coauthored than non-quantitative work. However, structural embeddedness within the network core given collaboration is largely unrelated to specialty area. This pattern is consistent with a loosely overlapping specialty structure that has potentially integrative implications for theoretical development in sociology."
1982829740,Rankings of Academic Journals and Institutions in Economics,"We conducted a worldwide ranking of academic institutions that produce research in a list of thirty top research journals in economics. We also computed journal rankings for the same period and hence we do not rely on weights that were computed for research carried out in earlier periods. The United States is clearly the dominant force in the top-fifty group, but European academic institutions are well represented in the group of the top 200 universities worldwide as are universities from Asia and the Far East in particular."
1713342896,What do we know about the h index?,"Abstract Jorge Hirsch (2005a, 2005b) recently proposed the h index to quantify the research output of individual scientists. The new index has attracted a lot of attention in the scientific community. The claim that the h index in a single number provides a good representation of the scientific lifetime achievement of a scientist as well as the (supposed) simple calculation of the h index using common literature databases lead to the danger of improper use of the index. We describe the advantages and disadvantages of the h index and summarize the studies on the convergent validity of this index. We also introduce corrections and complements as well as single‐number alternatives to the h index."
2518646621,Scholarly use of social media and altmetrics: A review of the literature,"Social media has become integrated into the fabric of the scholarly communication system in fundamental ways, principally through scholarly use of social media platforms and the promotion of new indicators on the basis of interactions with these platforms. Research and scholarship in this area has accelerated since the coining and subsequent advocacy for altmetrics—that is, research indicators based on social media activity. This review provides an extensive account of the state‐of‐the art in both scholarly use of social media and altmetrics. The review consists of 2 main parts: the first examines the use of social media in academia, reviewing the various functions these platforms have in the scholarly communication process and the factors that affect this use. The second part reviews empirical studies of altmetrics, discussing the various interpretations of altmetrics, data collection and methodological limitations, and differences according to platform. The review ends with a critical discussion of the implications of this transformation in the scholarly communication system."
2066752129,Atypical Combinations and Scientific Impact,"Novelty is an essential feature of creative ideas, yet the building blocks of new ideas are often embodied in existing knowledge. From this perspective, balancing atypical knowledge with conventional knowledge may be critical to the link between innovativeness and impact. Our analysis of 17.9 million papers spanning all scientific fields suggests that science follows a nearly universal pattern: The highest-impact science is primarily grounded in exceptionally conventional combinations of prior work yet simultaneously features an intrusion of unusual combinations. Papers of this type were twice as likely to be highly cited works. Novel combinations of prior work are rare, yet teams are 37.7% more likely than solo authors to insert novel combinations into familiar knowledge domains."
2765658614,The science of science: From the perspective of complex systems,"The science of science (SOS) is a rapidly developing field which aims to understand, quantify and predict scientific research and the resulting outcomes. The problem is essentially related to almost all scientific disciplines and thus has attracted attention of scholars from different backgrounds. Progress on SOS will lead to better solutions for many challenging issues, ranging from the selection of candidate faculty members by a university to the development of research fields to which a country should give priority. While different measurements have been designed to evaluate the scientific impact of scholars, journals and academic institutions, the multiplex structure, dynamics and evolution mechanisms of the whole system have been much less studied until recently. In this article, we review the recent advances in SOS, aiming to cover the topics from empirical study, network analysis, mechanistic models, ranking, prediction, and many important related issues. The results summarized in this review significantly deepen our understanding of the underlying mechanisms and statistical rules governing the science system. Finally, we review the forefront of SOS research and point out the specific difficulties as they arise from different contexts, so as to stimulate further efforts in this emerging interdisciplinary field."
2168201538,A Principal Component Analysis of 39 Scientific Impact Measures,"Background The impact of scientific publications has traditionally been expressed in terms of citation counts. However, scientific activity has moved online over the past decade. To better capture scientific impact in the digital era, a variety of new impact measures has been proposed on the basis of social network analysis and usage log data. Here we investigate how these new measures relate to each other, and how accurately and completely they express scientific impact. Methodology We performed a principal component analysis of the rankings produced by 39 existing and proposed measures of scholarly impact that were calculated on the basis of both citation and usage log data. Conclusions Our results indicate that the notion of scientific impact is a multi-dimensional construct that can not be adequately measured by any single indicator, although some measures are more suitable than others. The commonly used citation Impact Factor is not positioned at the core of this construct, but at its periphery, and should thus be used with caution."
2016860460,The Development of Open Access Journal Publishing from 1993 to 2009,"Open Access (OA) is a model for publishing scholarly peer reviewed journals, made possible by the Internet. The full text of OA journals and articles can be freely read, as the publishing is funded through means other than subscriptions. Empirical research concerning the quantitative development of OA publishing has so far consisted of scattered individual studies providing brief snapshots, using varying methods and data sources. This study adopts a systematic method for studying the development of OA journals from their beginnings in the early 1990s until 2009. Because no comprehensive index of OA articles exists, systematic manual data collection from journal web sites was conducted based on journal-level data extracted from the Directory of Open Access Journals (DOAJ). Due to the high number of journals registered in the DOAJ, almost 5000 at the time of the study, stratified random sampling was used. A separate sample of verified early pioneer OA journals was also studied. The results show a very rapid growth of OA publishing during the period 1993-2009. During the last year an estimated 191 000 articles were published in 4769 journals. Since the year 2000, the average annual growth rate has been 18% for the number of journals and 30% for the number of articles. This can be contrasted to the reported 3,5% yearly volume increase in journal publishing in general. In 2009 the share of articles in OA journals, of all peer reviewed journal articles, reached 7,7%. Overall, the results document a rapid growth in OA journal publishing over the last fifteen years. Based on the sampling results and qualitative data a division into three distinct periods is suggested: The Pioneering years (1993-1999), the Innovation years (2000-2004), and the Consolidation years (2005-2009)."
2048185449,"Self-Selected or Mandated, Open Access Increases Citation Impact for Higher Quality Research","Articles whose authors make them Open Access (OA) by self-archiving them online are cited significantly more than articles accessible only to subscribers. Some have suggested that this ""OA Advantage"" may not be causal but just a self-selection bias, because authors preferentially make higher-quality articles OA. To test this we compared self-selective self-archiving with mandatory self-archiving for a sample of 27,197 articles published 2002-2006 in 1,984 journals. The OA Advantage proved just as high for both. Logistic regression showed that the advantage is independent of other correlates of citations (article age; journal impact factor; number of co-authors, references or pages; field; article type; or country) and greatest for the most highly cited articles. The OA Advantage is real, independent and causal, but skewed. Its size is indeed correlated with quality, just as citations themselves are (the top 20% of articles receive about 80% of all citations). The advantage is greater for the more citeable articles, not because of a quality bias from authors self-selecting what to make OA, but because of a quality advantage, from users self-selecting what to use and cite, freed by OA from the constraints of selective accessibility to subscribers only."
2012950673,Peer review: a flawed process at the heart of science and journals,"Peer review is at the heart of the processes of not just medical journals but of all of science. It is the method by which grants are allocated, papers published, academics promoted, and Nobel prizes won. Yet it is hard to define. It has until recently been unstudied. And its defects are easier to identify than its attributes. Yet it shows no sign of going away. Famously, it is compared with democracy: a system full of problems but the least worst we have.When something is peer reviewed it is in some sense blessed. Even journalists recognize this. When the BMJ published a highly controversial paper that argued that a new `disease', female sexual dysfunction, was in some ways being created by pharmaceutical companies, a friend who is a journalist was very excited—not least because reporting it gave him a chance to get sex onto the front page of a highly respectable but somewhat priggish newspaper (the Financial Times). `But,' the news editor wanted to know, `was this paper peer reviewed?'. The implication was that if it had been it was good enough for the front page and if it had not been it was not. Well, had it been? I had read it much more carefully than I read many papers and had asked the author, who happened to be a journalist, to revise the paper and produce more evidence. But this was not peer review, even though I was a peer of the author and had reviewed the paper. Or was it? (I told my friend that it had not been peer reviewed, but it was too late to pull the story from the front page.)"
3125707221,Bibliometric Methods in Management and Organization,"We aim to develop a meaningful single-source reference for management and organization scholars interested in using bibliometric methods for mapping research specialties. Such methods introduce a measure of objectivity into the evaluation of scientific literature and hold the potential to increase rigor and mitigate researcher bias in reviews of scientific literature by aggregating the opinions of multiple scholars working in the field. We introduce the bibliometric methods of citation analysis, co-citation analysis, bibliographical coupling, co-author analysis, and co-word analysis and present a workflow for conducting bibliometric studies with guidelines for researchers. We envision that bibliometric methods will complement meta-analysis and qualitative structured literature reviews as a method for reviewing and evaluating scientific literature. To demonstrate bibliometric methods, we performed a citation and co-citation analysis to map the intellectual structure of the Organizational Research Methods journal."
2045816279,Google Scholar as a new source for citation analysis,"ESEP Ethics in Science and Environmental Politics Contact the journal RSS Mailing List Subscribe to our mailing list via Mailchimp HomeLatest VolumeAbout the JournalEditorsTheme Sections ESEP 8:61-73 (2008) - DOI: https://doi.org/10.3354/esep00076 Google Scholar as a new source for citation analysis Anne-Wil K. Harzing1,*, Ron van der Wal2 1Department of Management, University of Melbourne, Parkville Campus, Parkville, Victoria 3010, Australia 2Tarma Software Research, GPO Box 4063, Melbourne, Victoria 3001, Australia *Email: harzing@unimelb.edu.au ABSTRACT: Traditionally, the most commonly used source of bibliometric data is Thomson ISI Web of Knowledge, in particular the Web of Science and the Journal Citation Reports (JCR), which provide the yearly Journal Impact Factors (JIF). This paper presents an alternative source of data (Google Scholar, GS) as well as 3 alternatives to the JIF to assess journal impact (h-index, g-index and the number of citations per paper). Because of its broader range of data sources, the use of GS generally results in more comprehensive citation coverage in the area of management and international business. The use of GS particularly benefits academics publishing in sources that are not (well) covered in ISI. Among these are books, conference papers, non-US journals, and in general journals in the field of strategy and international business. The 3 alternative GS-based metrics showed strong correlations with the traditional JIF. As such, they provide academics and universities committed to JIFs with a good alternative for journals that are not ISI-indexed. However, we argue that these metrics provide additional advantages over the JIF and that the free availability of GS allows for a democratization of citation analysis as it provides every academic access to citation data regardless of their institution’s financial means. KEY WORDS: Google Scholar · Citation analysis · Publish or perish · h-index · g-index · Journal impact factor Full text in pdf format PreviousNextCite this article as: Harzing AWK, van der Wal R (2008) Google Scholar as a new source for citation analysis. Ethics Sci Environ Polit 8:61-73. https://doi.org/10.3354/esep00076 Export citation RSS - Facebook - Tweet - linkedIn Cited by Published in ESEP Vol. 8, No. 1. Online publication date: June 03, 2008 Print ISSN: 1863-5415; Online ISSN: ISSN Copyright © 2008 Inter-Research."
1715687586,Comparing the Impact of Open Access (OA) vs. Non-OA Articles in the Same Journals,The way to test the impact advantage of Open Access (OA) is not to compare the citation impact factors of OA and non-OA journals but to compare the citation counts of individual OA and non-OA articles appearing in the same (non-OA) journals. Such ongoing comparisons are revealing dramatic citation advantages for OA.
2163978340,"Journal Prestige, Publication Bias, and Other Characteristics Associated With Citation of Published Studies in Peer-Reviewed Journals","Citation by other authors is important in the dissemination of published science, but factors predicting it are little studied.To identify characteristics of published research predicting citation in other journals, we searched the Science Citations Index database for a standardized 3.5 years for all citations of published articles originally submitted to a 1991 emergency medicine specialty meeting. Analysis was conducted by classification and regression trees, a nonparametric modeling technique of regression trees, to determine the impact of previously determined characteristics of the full articles on the outcome measures. We calculated the the number of times an article was cited each year and calculated the mean impact factor (citations per manuscript per year) in other citing journals.Of the 493 submitted manuscripts, 204 published articles met entry criteria. The mean citations per year was 2.04 (95% confidence interval, 1.6-2.4; range, 0-20.9) in 440 different journals. Nineteen articles (9.3%) were never cited. The ability to predict the citations per year was weak (pseudo R(2) = 0.14.). The strongest predictor of citations per year was the impact factor of the original publishing journal. The presence of a control group, the subjective newsworthiness score, and sample size predicted citation frequency (24.3%, 26.0%, and 26.5% as strongly, respectively). The ability to predict mean impact factor of the citing journals was even weaker (pseudo R(2) = 0.09). The impact factor of the publishing journal was the strongest predictor, followed by the newsworthiness score (89.9% as strongly) and a subjective quality score (61.5%). Positive outcome bias was not evident for either outcome measure.In this cohort of published research, commonly used measures of study methodology and design did not predict the frequency of citations or the importance of citing journals. Positive outcome bias was not evident. The impact factor of the original publishing journal was more important than any other variable, suggesting that the journal in which a study is published may be as important as traditional measures of study quality in ensuring dissemination."
3148069922,The R- and AR-indices： Complementing the h-index,把 R-andAR 索引基于我们介绍并且学习的 h 索引打的基础。这些新索引消除一些 h 索引的劣势，特别当他们与 h 索引在联合被使用时。R 索引测量 h-core 的引证紧张，当 AR 去时进一步的一个步骤并且花出版物的很长时间进报道。这允许能实际上随着时间的过去增加并且减少的一个索引。我们建议对(h， AR ) 作为为研究评估的有意义的指示物。我们进一步证明尽最大努力描绘 h 索引的一种关系是法律模型。
1890493318,"Three options for citation tracking: Google Scholar, Scopus and Web of Science","Researchers turn to citation tracking to find the most influential articles for a particular topic and to see how often their own published papers are cited. For years researchers looking for this type of information had only one resource to consult: the Web of Science from Thomson Scientific. In 2004 two competitors emerged--Scopus from Elsevier and Google Scholar from Google. The research reported here uses citation analysis in an observational study examining these three databases; comparing citation counts for articles from two disciplines (oncology and condensed matter physics) and two years (1993 and 2003) to test the hypothesis that the different scholarly publication coverage provided by the three search tools will lead to different citation counts from each.Eleven journal titles with varying impact factors were selected from each discipline (oncology and condensed matter physics) using the Journal Citation Reports (JCR). All articles published in the selected titles were retrieved for the years 1993 and 2003, and a stratified random sample of articles was chosen, resulting in four sets of articles. During the week of November 7-12, 2005, the citation counts for each research article were extracted from the three sources. The actual citing references for a subset of the articles published in 2003 were also gathered from each of the three sources.For oncology 1993 Web of Science returned the highest average number of citations, 45.3. Scopus returned the highest average number of citations (8.9) for oncology 2003. Web of Science returned the highest number of citations for condensed matter physics 1993 and 2003 (22.5 and 3.9 respectively). The data showed a significant difference in the mean citation rates between all pairs of resources except between Google Scholar and Scopus for condensed matter physics 2003. For articles published in 2003 Google Scholar returned the largest amount of unique citing material for oncology and Web of Science returned the most for condensed matter physics.This study did not identify any one of these three resources as the answer to all citation tracking needs. Scopus showed strength in providing citing literature for current (2003) oncology articles, while Web of Science produced more citing material for 2003 and 1993 condensed matter physics, and 1993 oncology articles. All three tools returned some unique material. Our data indicate that the question of which tool provides the most complete set of citing literature may depend on the subject and publication year of a given article."
1986759740,"Comparisons of Citations in Web of Science, Scopus, and Google Scholar for Articles Published in General Medical Journals","ContextUntil recently, Web of Science was the only database available to track citation counts for published articles. Other databases are now available, but their relative performance has not been established.ObjectiveTo compare the citation count profiles of articles published in general medical journals among the citation databases of Web of Science, Scopus, and Google Scholar.DesignCohort study of 328 articles published in JAMA, Lancet, or the New England Journal of Medicine between October 1, 1999, and March 31, 2000. Total citation counts for each article up to June 2008 were retrieved from Web of Science, Scopus, and Google Scholar. Article characteristics were analyzed in linear regression models to determine interaction with the databases.Main Outcome MeasuresNumber of citations received by an article since publication and article characteristics associated with citation in databases.ResultsGoogle Scholar and Scopus retrieved more citations per article with a median of 160 (interquartile range [IQR], 83 to 324) and 149 (IQR, 78 to 289), respectively, than Web of Science (median, 122; IQR, 66 to 241) (P 10-transformed scale, fewer citations were found in Google Scholar to articles with declared industry funding (nonstandardized regression coefficient, −0.09; 95% confidence interval [CI], −0.15 to −0.03), reporting a study of a drug or medical device (−0.05; 95% CI, −0.11 to 0.01), or with group authorship (−0.29; 95% CI, −0.35 to −0.23). In multivariable analysis, group authorship was the only characteristic that differed among the databases; Google Scholar had significantly fewer citations to group-authored articles (−0.30; 95% CI, −0.36 to −0.23) compared with Web of Science.ConclusionWeb of Science, Scopus, and Google Scholar produced quantitatively and qualitatively different citation counts for articles published in 3 general medical journals."
2163887529,The metric tide: report of the independent review of the role of metrics in research assessment and management,"report presents the findings and recommendations of the Independent Review of the Role of Metrics in Research Assessment and Management. review was chaired by Professor James Wilsdon, supported by an independent and multidisciplinary group of experts in scientometrics, research funding, research policy, publishing, university management and administration. This review has gone beyond earlier studies to take a deeper look at potential uses and limitations of research metrics and indicators. It has explored the use of metrics across different disciplines, and assessed their potential contribution to the development of research excellence and impact. It has analysed their role in processes of research assessment, including the next cycle of the Research Excellence Framework (REF). It has considered the changing ways in which universities are using quantitative indicators in their management systems, and the growing power of league tables and rankings. And it has considered the negative or unintended effects of metrics on various aspects of research culture. The report starts by tracing the history of metrics in research management and assessment, in the UK and internationally. It looks at the applicability of metrics within different research cultures, compares the peer review system with metric-based alternatives, and considers what balance might be struck between the two. It charts the development of research management systems within institutions, and examines the effects of the growing use of quantitative indicators on different aspects of research culture, including performance management, equality, diversity, interdisciplinarity, and the ‘gaming’ of assessment systems. review looks at how different funders are using quantitative indicators, and considers their potential role in research and innovation policy. Finally, it examines the role that metrics played in REF2014, and outlines scenarios for their contribution to future exercises."
2108695571,"The e-Index, Complementing the h-Index for Excess Citations","The h-index has already been used by major citation databases to evaluate the academic performance of individual scientists. Although effective and simple, the h-index suffers from some drawbacks that limit its use in accurately and fairly comparing the scientific output of different researchers. These drawbacks include information loss and low resolution: the former refers to the fact that in addition to h(2) citations for papers in the h-core, excess citations are completely ignored, whereas the latter means that it is common for a group of researchers to have an identical h-index.To solve these problems, I here propose the e-index, where e(2) represents the ignored excess citations, in addition to the h(2) citations for h-core papers. Citation information can be completely depicted by using the h-index together with the e-index, which are independent of each other. Some other h-type indices, such as a and R, are h-dependent, have information redundancy with h, and therefore, when used together with h, mask the real differences in excess citations of different researchers.Although simple, the e-index is a necessary h-index complement, especially for evaluating highly cited scientists or for precisely comparing the scientific output of a group of scientists having an identical h-index."
2126566678,A new methodology for constructing a publication‐level classification system of science,"Classifying journals or publications into research areas is an essential element of many bibliometric analyses. Classification usually takes place at the level of journals, where the W eb of S cience subject categories are the most popular classification system. However, journal‐level classification systems have two important limitations: They offer only a limited amount of detail, and they have difficulties with multidisciplinary journals. To avoid these limitations, we introduce a new methodology for constructing classification systems at the level of individual publications. In the proposed methodology, publications are clustered into research areas based on citation relations. The methodology is able to deal with very large numbers of publications. We present an application in which a classification system is produced that includes almost 10 million publications. Based on an extensive analysis of this classification system, we discuss the strengths and the limitations of the proposed methodology. Important strengths are the transparency and relative simplicity of the methodology and its fairly modest computing and memory requirements. The main limitation of the methodology is its exclusive reliance on direct citation relations between publications. The accuracy of the methodology can probably be increased by also taking into account other types of relations–for instance, based on bibliographic coupling."
2124407761,Characteristics of highly cited papers,"Highly cited articles are very different from ‘ordinary’ cited articles. Typically, they are authored by a large number of scientists, often involving international collaboration. The majority of the papers represent regular journal articles (81%), although review articles (12%) are over-represented compared to the national average. The citation curves of highly cited papers follow a typical pattern of rise and decline. However, different types of citation curves can be identified, reflecting possible differences in the cognitive function of the articles. Highly cited papers typically obtain citations from a large number of different journals and from papers representing both close and remote fields. However, this pattern is not very different from the average distribution for all papers. We discuss how the findings can be explained by introducing a conceptual distinction between quality dynamics and visibility dynamics."
4237494090,Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine,"Abstract In this study, we examined empirical results on the h index and its most important variants in order to determine whether the variants developed are associated with an incremental contribution for evaluation purposes. The results of a factor analysis using bibliographic data on postdoctoral researchers in biomedicine indicate that regarding the h index and its variants, we are dealing with two types of indices that load on one factor each. One type describes the most productive core of a scientist's output and gives the number of papers in that core. The other type of indices describes the impact of the papers in the core. Because an index for evaluative purposes is a useful yardstick for comparison among scientists if the index corresponds strongly with peer assessments, we calculated a logistic regression analysis with the two factors resulting from the factor analysis as independent variables and peer assessment of the postdoctoral researchers as the dependent variable. The results of the regression analysis show that peer assessments can be predicted better using the factor ‘impact of the productive core’ than using the factor ‘quantity of the productive core.’"
2951923752,A global map of science based on the ISI subject categories,"Abstract The decomposition of scientific literature into disciplinary and subdisciplinary structures is one of the core goals of scientometrics. How can we achieve a good decomposition? The ISI subject categories classify journals included in the Science Citation Index (SCI). The aggregated journal‐journal citation matrix contained in the Journal Citation Reports can be aggregated on the basis of these categories. This leads to an asymmetrical matrix (citing versus cited) that is much more densely populated than the underlying matrix at the journal level. Exploratory factor analysis of the matrix of subject categories suggests a 14‐factor solution. This solution could be interpreted as the disciplinary structure of science. The nested maps of science (corresponding to 14 factors, 172 categories, and 6,164 journals) are online at http://www.leydesdorff.net/map06 . Presumably, inaccuracies in the attribution of journals to the ISI subject categories average out so that the factor analysis reveals the main structures. The mapping of science could, therefore, be comprehensive and reliable on a large scale albeit imprecise in terms of the attribution of journals to the ISI subject categories."
2140880926,Open Access to the Scientific Journal Literature: Situation 2009,"Background The Internet has recently made possible the free global availability of scientific journal articles. Open Access (OA) can occur either via OA scientific journals, or via authors posting manuscripts of articles published in subscription journals in open web repositories. So far there have been few systematic studies showing how big the extent of OA is, in particular studies covering all fields of science. Methodology/Principal Findings The proportion of peer reviewed scholarly journal articles, which are available openly in full text on the web, was studied using a random sample of 1837 titles and a web search engine. Of articles published in 2008, 8,5% were freely available at the publishers' sites. For an additional 11,9% free manuscript versions could be found using search engines, making the overall OA percentage 20,4%. Chemistry (13%) had the lowest overall share of OA, Earth Sciences (33%) the highest. In medicine, biochemistry and chemistry publishing in OA journals was more common. In all other fields author-posted manuscript copies dominated the picture. Conclusions/Significance The results show that OA already has a significant positive impact on the availability of the scientific journal literature and that there are big differences between scientific disciplines in the uptake. Due to the lack of awareness of OA-publishing among scientists in most fields outside physics, the results should be of general interest to all scholars. The results should also interest academic publishers, who need to take into account OA in their business strategies and copyright policies, as well as research funders, who like the NIH are starting to require OA availability of results from research projects they fund. The method and search tools developed also offer a good basis for more in-depth studies as well as longitudinal studies."
2121493052,Comparison of SCImago journal rank indicator with journal impact factor,"The application of currently available sophisticated algorithms of citation analysis allows for the incorporation of the ""quality"" of citations in the evaluation of scientific journals. We sought to compare the newly introduced SCImago journal rank (SJR) indicator with the journal impact factor (IF). We retrieved relevant information from the official Web sites hosting the above indices and their source databases. The SJR indicator is an open-access resource, while the journal IF requires paid subscription. The SJR indicator (based on Scopus data) lists considerably more journal titles published in a wider variety of countries and languages, than the journal IF (based on Web of Science data). Both indices divide citations to a journal by articles of the journal, during a specific time period. However, contrary to the journal IF, the SJR indicator attributes different weight to citations depending on the ""prestige"" of the citing journal without the influence of journal self-citations; prestige is estimated with the application of the PageRank algorithm in the network of journals. In addition, the SJR indicator includes the total number of documents of a journal in the denominator of the relevant calculation, whereas the journal IF includes only ""citable"" articles (mainly original articles and reviews). A 3-yr period is analyzed in both indices but with the use of different approaches. Regarding the top 100 journals in the 2006 journal IF ranking order, the median absolute change in their ranking position with the use of the SJR indicator is 32 (1st quartile: 12; 3rd quartile: 75). Although further validation is warranted, the novel SJR indicator poses as a serious alternative to the well-established journal IF, mainly due to its open-access nature, larger source database, and assessment of the quality of citations."
1970217003,"Open access publishing, article downloads, and citations: randomised controlled trial","Objective To measure the effect of free access to the scientific literature on article downloads and citations. Design Randomised controlled trial. Setting 11 journals published by the American Physiological Society. Participants 1619 research articles and reviews. Main outcome measures Article readership (measured as downloads of full text, PDFs, and abstracts) and number of unique visitors (internet protocol addresses). Citations to articles were gathered from the Institute for Scientific Information after one year. Interventions Random assignment on online publication of articles published in 11 scientific journals to open access (treatment) or subscription access (control). Results Articles assigned to open access were associated with 89% more full text downloads (95% confidence interval 76% to 103%), 42% more PDF downloads (32% to 52%), and 23% more unique visitors (16% to 30%), but 24% fewer abstract downloads (−29% to −19%) than subscription access articles in the first six months after publication. Open access articles were no more likely to be cited than subscription access articles in the first year after publication. Fifty nine per cent of open access articles (146 of 247) were cited nine to 12 months after publication compared with 63% (859 of 1372) of subscription access articles. Logistic and negative binomial regression analysis of article citation counts confirmed no citation advantage for open access articles. Conclusions Open access publishing may reach more readers than subscription access publishing. No evidence was found of a citation advantage for open access articles in the first year after publication. The citation advantage from open access reported widely in the literature may be an artefact of other causes."
2913788248,"Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories","Citations are increasingly used as performance indicators in research policy and within the research system. Usually, citations are assumed to reflect the impact of the research or its quality. What is the justification for these assumptions and how do citations relate to research quality? These and similar issues have been addressed through several decades of scientometric research. This article provides an overview of some of the main issues at stake, including theories of citation and the interpretation and validity of citations as performance measures. Research quality is a multidimensional concept, where plausibility/soundness, originality, scientific value, and societal value commonly are perceived as key characteristics. The article investigates how citations may relate to these various research quality dimensions. It is argued that citations reflect aspects related to scientific impact and relevance, although with important limitations. On the contrary, there is no evidence that citations reflect other key dimensions of research quality. Hence, an increased use of citation indicators in research evaluation and funding may imply less attention to these other research quality dimensions, such as solidity/plausibility, originality, and societal value."
4255497883,"Co‐citation analysis, bibliographic coupling, and direct citation: Which citation approach represents the research front most accurately?","Abstract In the past several years studies have started to appear comparing the accuracies of various science mapping approaches. These studies primarily compare the cluster solutions resulting from different similarity approaches, and give varying results. In this study we compare the accuracies of cluster solutions of a large corpus of 2,153,769 recent articles from the biomedical literature (2004–2008) using four similarity approaches: co‐citation analysis, bibliographic coupling, direct citation, and a bibliographic coupling‐based citation‐text hybrid approach. Each of the four approaches can be considered a way to represent the research front in biomedicine, and each is able to successfully cluster over 92% of the corpus. Accuracies are compared using two metrics—within‐cluster textual coherence as defined by the Jensen‐Shannon divergence, and a concentration measure based on the grant‐to‐article linkages indexed in MEDLINE. Of the three pure citation‐based approaches, bibliographic coupling slightly outperforms co‐citation analysis using both accuracy measures; direct citation is the least accurate mapping approach by far. The hybrid approach improves upon the bibliographic coupling results in all respects. We consider the results of this study to be robust given the very large size of the corpus, and the specificity of the accuracy measures used."
1519946736,The Case for Institutional Repositories: A SPARC Position Paper,"Editor's Note:  In July, the Scholarly Publishing and Academic Resources Coalition (SPARC) released a major position paper that examines the strategic roles institutional repositories serve for colleges and universities.  What follows are excerpts from the paper (footnotes omitted) that introduce the concept and describe the essential elements of an institutional repository.  The full paper is available on the SPARC Web site ."
2084091192,Hyperauthorship: A postmodern perversion or evidence of a structural shift in scholarly communication practices?,"Abstract Classical assumptions about the nature and ethical entailments of authorship (the standard model) are being challenged by developments in scientific collaboration and multiple authorship. In the biomedical research community, multiple authorship has increased to such an extent that the trustworthiness of the scientific communication system has been called into question. Documented abuses, such as honorific authorship, have serious implications in terms of the acknowledgment of authority, allocation of credit, and assigning of accountability. Within the biomedical world it has been proposed that authors be replaced by lists of contributors (the radical model), whose specific inputs to a given study would be recorded unambiguously. The wider implications of the ‘hyperauthorship’ phenomenon for scholarly publication are considered."
2002034968,Earlier Web usage statistics as predictors of later citation impact,"Abstract The use of citation counts to assess the impact of research articles is well established. However, the citation impact of an article can only be measured several years after it has been published. As research articles are increasingly accessed through the Web, the number of times an article is downloaded can be instantly recorded and counted. One would expect the number of times an article is read to be related both to the number of times it is cited and to how old the article is. The authors analyze how short‐term Web usage impact predicts medium‐term citation impact. The physics e‐print archive—arXiv.org—is used to test this."
2096523843,"Multi-University Research Teams: Shifting Impact, Geography, and Stratification in Science","This paper demonstrates that teamwork in science increasingly spans university boundaries, a dramatic shift in knowledge production that generalizes across virtually all fields of science, engineering, and social science. Moreover, elite universities play a dominant role in this shift. By examining 4.2 million papers published over three decades, we found that multi-university collaborations (i) are the fastest growing type of authorship structure, (ii) produce the highest-impact papers when they include a top-tier university, and (iii) are increasingly stratified by in-group university rank. Despite the rising frequency of research that crosses university boundaries, the intensification of social stratification in multi-university collaborations suggests a concentration of the production of scientific knowledge in fewer rather than more centers of high-impact science."
2057954853,The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index,"The growth rate of scientific publication has been studied from 1907 to 2007 using available data from a number of literature databases, including Science Citation Index (SCI) and Social Sciences Citation Index (SSCI). Traditional scientific publishing, that is publication in peer-reviewed journals, is still increasing although there are big differences between fields. There are no indications that the growth rate has decreased in the last 50 years. At the same time publication using new channels, for example conference proceedings, open archives and home pages, is growing fast. The growth rate for SCI up to 2007 is smaller than for comparable databases. This means that SCI was covering a decreasing part of the traditional scientific literature. There are also clear indications that the coverage by SCI is especially low in some of the scientific areas with the highest growth rate, including computer science and engineering sciences. The role of conference proceedings, open access archives and publications published on the net is increasing, especially in scientific fields with high growth rates, but this has only partially been reflected in the databases. The new publication channels challenge the use of the big databases in measurements of scientific productivity or output and of the growth rate of science. Because of the declining coverage and this challenge it is problematic that SCI has been used and is used as the dominant source for science indicators based on publication and citation numbers. The limited data available for social sciences show that the growth rate in SSCI was remarkably low and indicate that the coverage by SSCI was declining over time. National Science Indicators from Thomson Reuters is based solely on SCI, SSCI and Arts and Humanities Citation Index (AHCI). Therefore the declining coverage of the citation databases problematizes the use of this source."
2322381034,"The academic, economic and societal impacts of Open Access: an evidence-based review","Ongoing debates surrounding Open Access to the scholarly literature are multifaceted and complicated by disparate and often polarised viewpoints from engaged stakeholders. At the current stage, Open Access has become such a global issue that it is critical for all involved in scholarly publishing, including policymakers, publishers, research funders, governments, learned societies, librarians, and academic communities, to be well-informed on the history, benefits, and pitfalls of Open Access. In spite of this, there is a general lack of consensus regarding the potential pros and cons of Open Access at multiple levels. This review aims to be a resource for current knowledge on the impacts of Open Access by synthesizing important research in three major areas: academic, economic and societal. While there is clearly much scope for additional research, several key trends are identified, including a broad citation advantage for researchers who publish openly, as well as additional benefits to the non-academic dissemination of their work. The economic impact of Open Access is less well-understood, although it is clear that access to the research literature is key for innovative enterprises, and a range of governmental and non-governmental services. Furthermore, Open Access has the potential to save both publishers and research funders considerable amounts of financial resources, and can provide some economic benefits to traditionally subscription-based journals. The societal impact of Open Access is strong, in particular for advancing citizen science initiatives, and leveling the playing field for researchers in developing countries. Open Access supersedes all potential alternative modes of access to the scholarly literature through enabling unrestricted re-use, and long-term stability independent of financial constraints of traditional publishers that impede knowledge sharing. However, Open Access has the potential to become unsustainable for research communities if high-cost options are allowed to continue to prevail in a widely unregulated scholarly publishing market. Open Access remains only one of the multiple challenges that the scholarly publishing system is currently facing. Yet, it provides one foundation for increasing engagement with researchers regarding ethical standards of publishing and the broader implications of 'Open Research'."
2167872491,Co‐Authorship in Management and Organizational Studies: An Empirical and Network Analysis*,"In recent decades there has been growing interest in the nature and scale of scientific collaboration. Studies into co‐authorship have taken two different approaches. The first one attempts to analyse the reasons why authors collaborate and the consequences of such decision ( Laband and Tollison, 2000 ). The second approach is based on the idea that co‐authorship creates a social network of researchers ( Barabási et al., 2002 ; Moody, 2004 ; Newman, 2001 ). In this study we have carried out an exploratory analysis of co‐authorships in the field of management from the two aforementioned approaches. The results obtained show a growing tendency of the co‐authored papers in the field of management, similar to what can be observed in other disciplines. Our study analyses some of the underpinning factors, which have been highlighted in the literature, explaining this tendency. Thus, the progressive quantitative character of research and the influence of the collaboration on the articles' impact are enhanced. The network analysis permits the exploration of the peculiarities of the management in comparison with other fields of knowledge, as well as the existing linkages between the most central and prominent authors within this discipline."
2110158660,"A general framework for analysing diversity in science, technology and society","This paper addresses the scope for more integrated general analysis of diversity in science, technology and society. It proposes a framework recognizing three necessary but individually insufficient properties of diversity. Based on 10 quality criteria, it suggests a general quantitative non-parametric diversity heuristic. This allows the systematic exploration of diversity under different perspectives, including divergent conceptions of relevant attributes and contrasting weightings on different diversity properties. It is shown how this heuristic may be used to explore different possible trade-offs between diversity and other aspects of interest, including portfolio interactions. The resulting approach offers a way to be more systematic and transparent in the treatment of scientific and technological diversity in a range of fields, including conservation management, research governance, energy policy and sustainable innovation."
1491754772,Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature,"Data collected by social media platforms have been introduced as new sources for indicators to help measure the impact of scholarly research in ways that are complementary to traditional citation analysis. Data generated from social media activities can be used to reflect broad types of impact. This article aims to provide systematic evidence about how often Twitter is used to disseminate information about journal articles in the biomedical sciences. The analysis is based on 1.4 million documents covered by both PubMed and Web of Science and published between 2010 and 2012. The number of tweets containing links to these documents was analyzed and compared to citations to evaluate the degree to which certain journals, disciplines, and specialties were represented on Twitter and how far tweets correlate with citation impact. With less than 10% of PubMed articles mentioned on Twitter, its uptake is low in general but differs between journals and specialties. Correlations between tweets and citations are low, implying that impact metrics based on tweets are different from those based on citations. A framework using the coverage of articles and the correlation between Twitter mentions and citations is proposed to facilitate the evaluation of novel social‐media‐based metrics."
2547045893,Quantifying the evolution of individual scientific impact,"Despite the frequent use of numerous quantitative indicators to gauge the professional impact of a scientist, little is known about how scientific impact emerges and evolves in time. Here, we quantify the changes in impact and productivity throughout a career in science, finding that impact, as measured by influential publications, is distributed randomly within a scientist's sequence of publications. This random-impact rule allows us to formulate a stochastic model that uncouples the effects of productivity, individual ability, and luck and unveils the existence of universal patterns governing the emergence of scientific success.The model assigns a unique individual parameter Q to each scientist, which is stable during a career, and it accurately predicts the evolution of a scientist's impact, from the h-index to cumulative citations, and independent recognitions, such as prizes."
1575605140,Impact factor: a valid measure of journal quality?,"Impact factor, an index based on the frequency with which a journal's articles are cited in scientific publications, is a putative marker of journal quality. However, empiric studies on impact factor's validity as an indicator of quality are lacking. The authors assessed the validity of impact factor as a measure of quality for general medical journals by testing its association with journal quality as rated by clinical practitioners and researchers.We surveyed physicians specializing in internal medicine in the United States, randomly sampled from the American Medical Association's Physician Masterfile (practitioner group, n = 113) and from a list of graduates from a national postdoctoral training program in clinical and health services research (research group, n = 151). Respondents rated the quality of nine general medical journals, and we assessed the correlation between these ratings and the journals' impact factors.The correlation between impact factor and physicians' ratings of journal quality was strong (r2 = 0.82, P = 0.001). The correlation was higher for the research group (r2 = 0.83, P = 0.001) than for the practitioner group (r2 = 0.62, P = 0.01).Impact factor may be a reasonable indicator of quality for general medical journals."
2023783748,Open access versus subscription journals: a comparison of scientific impact,"In the past few years there has been an ongoing debate as to whether the proliferation of open access (OA) publishing would damage the peer review system and put the quality of scientific journal publishing at risk. Our aim was to inform this debate by comparing the scientific impact of OA journals with subscription journals, controlling for journal age, the country of the publisher, discipline and (for OA publishers) their business model. The 2-year impact factors (the average number of citations to the articles in a journal) were used as a proxy for scientific impact. The Directory of Open Access Journals (DOAJ) was used to identify OA journals as well as their business model. Journal age and discipline were obtained from the Ulrich's periodicals directory. Comparisons were performed on the journal level as well as on the article level where the results were weighted by the number of articles published in a journal. A total of 610 OA journals were compared with 7,609 subscription journals using Web of Science citation data while an overlapping set of 1,327 OA journals were compared with 11,124 subscription journals using Scopus data. Overall, average citation rates, both unweighted and weighted for the number of articles per journal, were about 30% higher for subscription journals. However, after controlling for discipline (medicine and health versus other), age of the journal (three time periods) and the location of the publisher (four largest publishing countries versus other countries) the differences largely disappeared in most subcategories except for journals that had been launched prior to 1996. OA journals that fund publishing with article processing charges (APCs) are on average cited more than other OA journals. In medicine and health, OA journals founded in the last 10 years are receiving about as many citations as subscription journals launched during the same period. Our results indicate that OA journals indexed in Web of Science and/or Scopus are approaching the same scientific impact and quality as subscription journals, particularly in biomedicine and for journals funded by article processing charges."
1021000864,The bibliometric analysis of scholarly production: How great is the impact?,"Bibliometric methods or “analysis” are now firmly established as scientific specialties and are an integral part of research evaluation methodology especially within the scientific and applied fields. The methods are used increasingly when studying various aspects of science and also in the way institutions and universities are ranked worldwide. A sufficient number of studies have been completed, and with the resulting literature, it is now possible to analyse the bibliometric method by using its own methodology. The bibliometric literature in this study, which was extracted from Web of Science, is divided into two parts using a method comparable to the method of Jonkers et al. (Characteristics of bibliometrics articles in library and information sciences (LIS) and other journals, pp. 449–551, 2012: The publications either lie within the Information and Library Science (ILS) category or within the non-ILS category which includes more applied, “subject” based studies. The impact in the different groupings is judged by means of citation analysis using normalized data and an almost linear increase can be observed from 1994 onwards in the non-ILS category. The implication for the dissemination and use of the bibliometric methods in the different contexts is discussed. A keyword analysis identifies the most popular subjects covered by bibliometric analysis, and multidisciplinary articles are shown to have the highest impact. A noticeable shift is observed in those countries which contribute to the pool of bibliometric analysis, as well as a self-perpetuating effect in giving and taking references."
2153511600,"The Leiden ranking 2011/2012: Data collection, indicators, and interpretation","The Leiden Ranking 2011/2012 is a ranking of universities based on bibliometric indicators of publication output, citation impact, and scientific collaboration. The ranking includes 500 major universities from 41 different countries. This paper provides an extensive discussion of the Leiden Ranking 2011/2012. The ranking is compared with other global university rankings, in particular the Academic Ranking of World Universities (commonly known as the Shanghai Ranking) and the Times Higher Education World University Rankings. The comparison focuses on the methodological choices underlying the different rankings. Also, a detailed description is offered of the data collection methodology of the Leiden Ranking 2011/2012 and of the indicators used in the ranking. Various innovations in the Leiden Ranking 2011/2012 are presented. These innovations include (1) an indicator based on counting a university's highly cited publications, (2) indicators based on fractional rather than full counting of collaborative publications, (3) the possibility of excluding non‐English language publications, and (4) the use of stability intervals. Finally, some comments are made on the interpretation of the ranking and a number of limitations of the ranking are pointed out."
2089745446,The Access/Impact Problem and the Green and Gold Roads to Open Access,"The research access/impact problem arises because journal articles are not accessible to all of their would-be users; hence, they are losing potential research impact. The solution is to make all articles Open Access (OA; i.e., accessible online, free for all). OA articles have significantly higher citation impact than non-OA articles. There are two roads to OA: the “golden” road (publish your article in an OA journal) and the “green” road (publish your article in a non-OA journal but also self-archive it in an OA archive). Only 5% of journals are gold, but over 90% are already green (i.e., they have given their authors the green light to self-archive); yet only about 10–20% of articles have been self-archived. To reach 100% OA, self-archiving needs to be mandated by researchers' employers and funders, as the United Kingdom and the United States have recently recommended, and universities need to implement that mandate."
2127775997,"Social network analysis: a powerful strategy, also for the information sciences","Social network analysis (SNA) is not a formal theory in sociology but rather a strategy for investigating social structures. As it is an idea that can be applied in many fields, we study, in particular, its influence in the information sciences. Information scientists study publication, citation and co-citation networks, collaboration structures and other forms of social interaction networks. Moreover, the Internet represents a social network of an unprecedented scale. In all these studies social network analysis can successfully be applied. SNA is further related to recent theories concerning the free market economy, geography and transport networks. The growth of SNA is documented and a co-author network of SNA is drawn. Centrality measures of the SNA network are calculated."
1907286193,Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references,"Many studies (in information science) have looked at the growth of science. In this study, we reexamine the question of the growth of science. To do this we (a) use current data up to publication year 2012 and (b) analyze the data across all disciplines and also separately for the natural sciences and for the medical and health sciences. Furthermore, the data were analyzed with an advanced statistical technique—segmented regression analysis—which can identify specific segments with similar growth rates in the history of science. The study is based on two different sets of bibliometric data: (a) the number of publications held as source items in the Web of Science ( WoS , T homson R euters) per publication year and (b) the number of cited references in the publications of the source items per cited reference year. We looked at the rate at which science has grown since the mid‐1600s. In our analysis of cited references we identified three essential growth phases in the development of science, which each led to growth rates tripling in comparison with the previous phase: from less than 1% up to the middle of the 18th century, to 2 to 3% up to the period between the two world wars, and 8 to 9% to 2010."
3038100267,Journal status,"The status of an actor in a social context is commonly defined in terms of two factors: the total number of endorsements the actor receives from other actors and the prestige of the endorsing actors. These two factors indicate the distinction between popularity and expert appreciation of the actor, respectively. We refer to the former as popularity and to the latter as prestige. These notions of popularity and prestige also apply to the domain of scholarly assessment. The ISI Impact Factor (ISI IF) is defined as the mean number of citations a journal receives over a 2 year period. By merely counting the amount of citations and disregarding the prestige of the citing journals, the ISI IF is a metric of popularity, not of prestige. We demonstrate how a weighted version of the popular PageRank algorithm can be used to obtain a metric that reflects prestige. We contrast the rankings of journals according to their ISI IF and their Weighted PageRank, and we provide an analysis that reveals both significant overlaps and differences. Furthermore, we introduce the Y-factor which is a simple combination of both the ISI IF and the weighted PageRank, and find that the resulting journal rankings correspond well to a general understanding of journal status."
3123879035,The Slowdown of the Economics Publishing Process,"Over the last three decades there has been a dramatic slowdown of the publication process at top economics journals. A substantial part is due to journals' requiring more extensive revisions. Various explanations are considered: democratization of the review process, increases in the complexity of papers, growth of the profession, and cost and benefit arguments. Changes in the profession are examined using time‐series data. Connections between these changes and the slowdown are examined using paper‐level data. There is evidence for some explanations, but most of the slowdown remains unexplained. Changes may reflect evolving social norms."
2130247873,The inconsistency of the h‐index,"Abstract The h‐index is a popular bibliometric indicator for assessing individual scientists. We criticize the h‐index from a theoretical point of view. We argue that for the purpose of measuring the overall scientific impact of a scientist (or some other unit of analysis), the h‐index behaves in a counterintuitive way. In certain cases, the mechanism used by the h‐index to aggregate publication and citation statistics into a single number leads to inconsistencies in the way in which scientists are ranked. Our conclusion is that the h‐index cannot be considered an appropriate indicator of a scientist's overall scientific impact. Based on recent theoretical insights, we discuss what kind of indicators can be used as an alternative to the h‐index. We pay special attention to the highly cited publications indicator. This indicator has a lot in common with the h‐index, but unlike the h‐index it does not produce inconsistent rankings."
2952533362,A review of the literature on citation impact indicators,"Citation impact indicators nowadays play an important role in research evaluation, and consequently these indicators have received a lot of attention in the bibliometric and scientometric literature. This paper provides an in-depth review of the literature on citation impact indicators. First, an overview is given of the literature on bibliographic databases that can be used to calculate citation impact indicators (Web of Science, Scopus, and Google Scholar). Next, selected topics in the literature on citation impact indicators are reviewed in detail. The first topic is the selection of publications and citations to be included in the calculation of citation impact indicators. The second topic is the normalization of citation impact indicators, in particular normalization for field differences. Counting methods for dealing with co-authored publications are the third topic, and citation impact indicators for journals are the last topic. The paper concludes by offering some recommendations for future research."
2235214564,Evaluation practices and effects of indicator use—a literature review,"Journal Article Evaluation practices and effects of indicator use—a literature review Get access Sarah de Rijcke, Sarah de Rijcke * 1 Centre for Science and Technology Studies (CWTS), Faculty of Social Sciences, Leiden University, 2300 AX, The Netherlands, *Corresponding author. Email: s.de.rijcke@cwts.leidenuniv.nl Search for other works by this author on: Oxford Academic Google Scholar Paul F. Wouters, Paul F. Wouters 1 Centre for Science and Technology Studies (CWTS), Faculty of Social Sciences, Leiden University, 2300 AX, The Netherlands, Search for other works by this author on: Oxford Academic Google Scholar Alex D. Rushforth, Alex D. Rushforth 1 Centre for Science and Technology Studies (CWTS), Faculty of Social Sciences, Leiden University, 2300 AX, The Netherlands, Search for other works by this author on: Oxford Academic Google Scholar Thomas P. Franssen, Thomas P. Franssen 1 Centre for Science and Technology Studies (CWTS), Faculty of Social Sciences, Leiden University, 2300 AX, The Netherlands, Search for other works by this author on: Oxford Academic Google Scholar Björn Hammarfelt Björn Hammarfelt 1 Centre for Science and Technology Studies (CWTS), Faculty of Social Sciences, Leiden University, 2300 AX, The Netherlands, 2 Swedish School of Library and Information Science (SSLIS), University of Borås, 501 90, Sweden Search for other works by this author on: Oxford Academic Google Scholar Research Evaluation, Volume 25, Issue 2, April 2016, Pages 161–169, https://doi.org/10.1093/reseval/rvv038 Published: 29 December 2015"
2053964603,What do we measure by co-authorships?,"Interviews with scientists about the content and reward of collaborations, and classification of contributions of co-authors and scientists cited in acknowledgements, identified six types of research collaborations with distinct patterns of rewards; showed that about half of the collaborations are invisible in formal communication channels because they are not rewarded; and showed that about one third of the collaborations are rewarded only by acknowledgements."
2033079808,Citation counts for research evaluation: standards of good practice for analyzing bibliometric data and presenting and interpreting results,"ESEP Ethics in Science and Environmental Politics Contact the journal RSS Mailing List Subscribe to our mailing list via Mailchimp HomeLatest VolumeAbout the JournalEditorsTheme Sections ESEP 8:93-102 (2008) - DOI: https://doi.org/10.3354/esep00084 Citation counts for research evaluation: standards of good practice for analyzing bibliometric data and presenting and interpreting results Lutz Bornmann1,*, Rüdiger Mutz1, Christoph Neuhaus1, Hans-Dieter Daniel1,2 1ETH Zurich, Professorship for Social Psychology and Research on Higher Education, Zähringerstr. 24, 8092 Zurich, Switzerland 2University of Zurich, Evaluation Office, Mühlegasse 21, 8001 Zurich, Switzerland *Email: bornmann@gess.ethz.ch ABSTRACT: With the ready accessibility of bibliometric data and the availability of ready-to-use tools for generating bibliometric indicators for evaluation purposes, there is the danger of inappropriate use. Here we present standards of good practice for analyzing bibliometric data and presenting and interpreting the results. Comparisons drawn between research groups as to research performance are valid only if (1) the scientific impact of the research groups or their publications are looked at by using box plots, Lorenz curves, and Gini coefficients to represent distribution characteristics of data (in other words, going beyond the usual arithmetic mean value), (2) different reference standards are used to assess the impact of research groups, and the appropriateness of the reference standards undergoes critical examination, and (3) statistical analyses comparing citation counts take into consideration that citations are a function of many influencing factors besides scientific quality. KEY WORDS: Citation counts · Research evaluation · Citation distribution · Reference standards · Lorenz curve · Gini coefficient · Box plots Full text in pdf format PreviousNextCite this article as: Bornmann L, Mutz R, Neuhaus C, Daniel HD (2008) Citation counts for research evaluation: standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics Sci Environ Polit 8:93-102. https://doi.org/10.3354/esep00084 Export citation RSS - Facebook - Tweet - linkedIn Cited by Published in ESEP Vol. 8, No. 1. Online publication date: June 03, 2008 Print ISSN: 1863-5415; Online ISSN: ISSN Copyright © 2008 Inter-Research."
1958739611,The Eigenfactor™ Metrics: Figure 1.,"Quantitative metrics are poor choices for assessing the research output of an individual scholar. Summing impact factors, counting citations, tallying an h-index, or looking at Eigenfactor™ Scores (described below)—none of these methods are adequate compared with what should be the gold standard"
1979116104,What is societal impact of research and how can it be assessed? a literature survey,"Since the 1990s, the scope of research evaluations becomes broader as the societal products (outputs), societal use (societal references), and societal benefits (changes in society) of research come into scope. Society can reap the benefits of successful research studies only if the results are converted into marketable and consumable products (e.g., medicaments, diagnostic tools, machines, and devices) or services. A series of different names have been introduced which refer to the societal impact of research: third stream activities, societal benefits, societal quality, usefulness, public values, knowledge transfer, and societal relevance. What most of these names are concerned with is the assessment of social, cultural, environmental, and economic returns (impact and effects) from results (research output) or products (research outcome) of publicly funded research. This review intends to present existing research on and practices employed in the assessment of societal impact in the form of a literature survey. The objective is for this review to serve as a basis for the development of robust and reliable methods of societal impact measurement."
3101807887,An index to quantify an individual's scientific research valid across disciplines,"The number h of papers with at least h citations has been proposed to evaluate individual's scientific research production. This index is robust in several ways but yet strongly dependent on the research field. We propose a complementary index hI = h^2/N_t, with N_t being the total number of authors in the considered h papers. A researcher with index hI has hI papers with at least hI citation if he/she had published alone. We have obtained the rank plots of h and hI for four Brazilian scientific communities. Contrasting to the h-index curve, the hI index present a perfect data collapse into a unique allowing comparison among different research areas."
2066520537,The Role of Gender in Scholarly Authorship,"Gender disparities appear to be decreasing in academia according to a number of metrics, such as grant funding, hiring, acceptance at scholarly journals, and productivity, and it might be tempting to think that gender inequity will soon be a problem of the past. However, a large-scale analysis based on over eight million papers across the natural sciences, social sciences, and humanities re- reveals a number of understated and persistent ways in which gender inequities remain. For instance, even where raw publication counts seem to be equal between genders, close inspection reveals that, in certain fields, men predominate in the prestigious first and last author positions. Moreover, women are significantly underrepresented as authors of single-authored papers. Academics should be aware of the subtle ways that gender disparities can appear in scholarly authorship."
3137875885,Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World,"Nowadays, the importance of bibliographic databases (DBs) has increased enormously, as they are the main providers of publication metadata and bibliometric indicators universally used both for research assessment practices and for performing daily tasks. Because the reliability of these tasks firstly depends on the data source, all users of the DBs should be able to choose the most suitable one. Web of Science (WoS) and Scopus are the two main bibliographic DBs. The comprehensive evaluation of the DBs’ coverage is practically impossible without extensive bibliometric analyses or literature reviews, but most DBs users do not have bibliometric competence and/or are not willing to invest additional time for such evaluations. Apart from that, the convenience of the DB’s interface, performance, provided impact indicators and additional tools may also influence the users’ choice. The main goal of this work is to provide all of the potential users with an all-inclusive description of the two main bibliographic DBs by gathering the findings that are presented in the most recent literature and information provided by the owners of the DBs at one place. This overview should aid all stakeholders employing publication and citation data in selecting the most suitable DB."
2160441315,Searching for intellectual turning points: Progressive knowledge domain visualization,"This article introduces a previously undescribed method progressively visualizing the evolution of a knowledge domain's cocitation network. The method first derives a sequence of cocitation networks from a series of equal-length time interval slices. These time-registered networks are merged and visualized in a panoramic view in such a way that intellectually significant articles can be identified based on their visually salient features. The method is applied to a cocitation study of the superstring field in theoretical physics. The study focuses on the search of articles that triggered two superstring revolutions. Visually salient nodes in the panoramic view are identified, and the nature of their intellectual contributions is validated by leading scientists in the field. The analysis has demonstrated that a search for intellectual turning points can be narrowed down to visually salient nodes in the visualized network. The method provides a promising way to simplify otherwise cognitively demanding tasks to a search for landmarks, pivots, and hubs."
2953304564,Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level,"Despite their recognized limitations, bibliometric assessments of scientific productivity have been widely adopted. We describe here an improved method to quantify the influence of a research article by making novel use of its co-citation network to field-normalize the number of citations it has received. Article citation rates are divided by an expected citation rate that is derived from performance of articles in the same field and benchmarked to a peer comparison group. The resulting Relative Citation Ratio is article level and field independent and provides an alternative to the invalid practice of using journal impact factors to identify influential papers. To illustrate one application of our method, we analyzed 88,835 articles published between 2003 and 2010 and found that the National Institutes of Health awardees who authored those papers occupy relatively stable positions of influence across all disciplines. We demonstrate that the values generated by this method strongly correlate with the opinions of subject matter experts in biomedical research and suggest that the same approach should be generally applicable to articles published in all areas of science. A beta version of iCite, our web tool for calculating Relative Citation Ratios of articles listed in PubMed, is available at https://icite.od.nih.gov."
2135547856,When Knowledge Wins: Transcending the Sense and Nonsense of Academic Rankings,"“Not everything that can be counted counts, and not everything that counts can be counted.” —Albert Einstein Has university scholarship gone astray? Do our academic assessment systems reward scholarship that addresses the questions that matter most to society? Using international business as an example, we highlight the problematic nature of academic ranking systems and question if such assessments are drawing scholarship away from its fundamental purpose. We call for an immediate examination of existing ranking systems, not only as a legitimate scholarly question vis-a-vis performance—a conceptual lens with deep roots in management research—but also because the very health and vibrancy of the field are at stake. Indeed, in light of the data presented here, which suggest that current systems are dysfunctional and potentially cause more harm than good, a temporary moratorium on rankings may be appropriate until more valid and reliable ways to assess scholarly contributions can be developed. The worldwide community of scholars, along with the global network of institutions interacting with and supporting management scholarship (such as the Academy of Management, AACSB, and Thomson Reuters Scientific) are invited to innovate and design more reliable and valid ways to assess scholarly contributions that truly promote the advancement of relevant 21st century knowledge, and likewise recognize those individuals and institutions that best fulfill the university's fundamental purpose."
2160500356,"Revealed Performances: Worldwide Rankings of Economists and Economics Departments, 1990–2000","In this paper, I study the production of academic research by economics departments and economists. Worldwide rankings are provided based on both citations and publications. These rankings reveal a dominant position of the United States in the production of economics literature. Over time, however, the extent of this dominance is decreasing."
2067333003,Coercive Citation in Academic Publishing,Many journal editors appear to strategically target authors and papers to pressure them into citing the editors' journals.
2122130843,Scientometrics 2.0: New metrics of scholarly impact on the social Web,"The growing flood of scholarly literature is exposing the weaknesses of current, citation-based methods of evaluating and filtering articles. A novel and promising approach is to examine the use and citation of articles in a new forum: Web 2.0 services like social bookmarking and microblogging. Metrics based on this data could build a “Scientometics 2.0,” supporting richer and more timely pictures of articles' impact. This paper develops the most comprehensive list of these services to date, assessing the potential value and availability of data from each. We also suggest the next steps toward building and validating metrics drawn from the social Web."
4388150685,The Intellectual and Social Organization Of the Sciences,"Abstract In a rapidly changing and inter-disciplinary world it is important to understand the nature and generation of knowledge, and its social organization. Increasing attention is paid in the social sciences and management studies to the constitution and claims of different theories, perspectives, and 'paradigms'. This book is one of the most respected and robust analyses of these issues. For this new paperback edition Richard Whitley - a leading figure in European business education - has written a new introduction which addresses the particular epistemological issues presented by management and business studies. He approaches the sciences as differently organized systems for the production and validation of knowledge - systems which become established in particular contexts and which generate different sorts of knowledge. He identifies seven major types of scientific field and discusses the establishment and growth of these sciences, including the major consequences of the nineteenth-century expansion of employment opportunities for researchers; the competitive pursuit of public reputations; and the domination of intellectual work by employees. He also examines the divergences in the way research is organized and controlled both in different fields, and in the same field within different historical circumstances. This book will be of interest to all graduate students concerned with the social study of knowledge, science, technology, and the history and philosophy of science."
2147854962,Betweenness centrality as an indicator of the interdisciplinarity of scientific journals,"Abstract In addition to science citation indicators of journals like impact and immediacy, social network analysis provides a set of centrality measures like degree, betweenness, and closeness centrality. These measures are first analyzed for the entire set of 7,379 journals included in the Journal Citation Reports of the Science Citation Index and the Social Sciences Citation Index 2004 (Thomson ISI, Philadelphia, PA), and then also in relation to local citation environments that can be considered as proxies of specialties and disciplines. Betweenness centrality is shown to be an indicator of the interdisciplinarity of journals, but only in local citation environments and after normalization; otherwise, the influence of degree centrality (size) overshadows the betweenness‐centrality measure. The indicator is applied to a variety of citation environments, including policy‐relevant ones like biotechnology and nanotechnology. The values of the indicator remain sensitive to the delineations of the set because of the indicator's local character. Maps showing interdisciplinarity of journals in terms of betweenness centrality can be drawn using information about journal citation environments, which is available online."
2056179849,"Mapping world scientific collaboration: Authors, institutions, and countries","Abstract International collaboration is being heralded as the hallmark of contemporary scientific production. Yet little quantitative evidence has portrayed the landscape and trends of such collaboration. To this end, 14,000,000 documents indexed in Thomson Reuters's Web of Science (WoS) were studied to provide a state‐of‐the‐art description of scientific collaborations across the world. The results indicate that the number of authors in the largest research teams have not significantly grown during the past decade; however, the number of smaller research teams has seen significant increases in growth. In terms of composition, the largest teams have become more diverse than the latter teams and tend more toward interinstitutional and international collaboration. Investigating the size of teams showed large variation between fields. Mapping scientific cooperation at the country level reveals that Western countries situated at the core of the map are extensively cooperating with each other. High‐impact institutions are significantly more collaborative than others. This work should inform policy makers, administrators, and those interested in the progression of scientific collaboration."
2585591443,Potential predatory and legitimate biomedical journals: can you tell the difference? A cross-sectional comparison,"The Internet has transformed scholarly publishing, most notably, by the introduction of open access publishing. Recently, there has been a rise of online journals characterized as 'predatory', which actively solicit manuscripts and charge publications fees without providing robust peer review and editorial services. We carried out a cross-sectional comparison of characteristics of potential predatory, legitimate open access, and legitimate subscription-based biomedical journals.On July 10, 2014, scholarly journals from each of the following groups were identified - potential predatory journals (source: Beall's List), presumed legitimate, fully open access journals (source: PubMed Central), and presumed legitimate subscription-based (including hybrid) journals (source: Abridged Index Medicus). MEDLINE journal inclusion criteria were used to screen and identify biomedical journals from within the potential predatory journals group. One hundred journals from each group were randomly selected. Journal characteristics (e.g., website integrity, look and feel, editors and staff, editorial/peer review process, instructions to authors, publication model, copyright and licensing, journal location, and contact) were collected by one assessor and verified by a second. Summary statistics were calculated.Ninety-three predatory journals, 99 open access, and 100 subscription-based journals were analyzed; exclusions were due to website unavailability. Many more predatory journals' homepages contained spelling errors (61/93, 66%) and distorted or potentially unauthorized images (59/93, 63%) compared to open access journals (6/99, 6% and 5/99, 5%, respectively) and subscription-based journals (3/100, 3% and 1/100, 1%, respectively). Thirty-one (33%) predatory journals promoted a bogus impact metric - the Index Copernicus Value - versus three (3%) open access journals and no subscription-based journals. Nearly three quarters (n = 66, 73%) of predatory journals had editors or editorial board members whose affiliation with the journal was unverified versus two (2%) open access journals and one (1%) subscription-based journal in which this was the case. Predatory journals charge a considerably smaller publication fee (median $100 USD, IQR $63-$150) than open access journals ($1865 USD, IQR $800-$2205) and subscription-based hybrid journals ($3000 USD, IQR $2500-$3000).We identified 13 evidence-based characteristics by which predatory journals may potentially be distinguished from presumed legitimate journals. These may be useful for authors who are assessing journals for possible submission or for others, such as universities evaluating candidates' publications as part of the hiring process."
2130456849,Team size matters: Collaboration and scientific impact since 1900,"This article provides the first historical analysis of the relationship between collaboration and scientific impact using three indicators of collaboration (number of authors, number of addresses, and number of countries) derived from articles published between 1900 and 2011. The results demonstrate that an increase in the number of authors leads to an increase in impact, from the beginning of the last century onward, and that this is not due simply to self‐citations. A similar trend is also observed for the number of addresses and number of countries represented in the byline of an article. However, the constant inflation of collaboration since 1900 has resulted in diminishing citation returns: Larger and more diverse (in terms of institutional and country affiliation) teams are necessary to realize higher impact. The article concludes with a discussion of the potential causes of the impact gain in citations of collaborative papers."
2088037265,"Changes in the intellectual structure of strategic management research: a bibliometric study of the Strategic Management Journal, 1980–2000","Abstract The aim of this paper is to identify the works that have had the greatest impact on strategic management research and to analyze the changes that have taken place in the intellectual structure of this discipline. The methodology is based on the bibliometric techniques of citation and co‐citation analysis which are applied to all the articles published in the Strategic Management Journal from its first issue in 1980 through 2000. Copyright © 2004 John Wiley & Sons, Ltd."
1964036498,"A Systematic Review of Research on the Meaning, Ethics and Practices of Authorship across Scholarly Disciplines","The purpose of this systematic review was to evaluate evidence about authorship issues and provide synthesis of research on authorship across all research fields.We searched bibliographical databases to identify articles describing empirical quantitive or qualitative research from all scholarly fields on different aspects of authorship. Search was limited to original articles and reviews.The final sample consisted of 123 articles reporting results from 118 studies. Most studies came for biomedical and health research fields and social sciences. Study design was usually a survey (53%) or descriptive study (27%); only 2 studies used randomized design. We identified four 4 general themes common to all research disciplines: authorship perceptions, definitions and practices, defining order of authors on the byline, ethical and unethical authorship practices, and authorship issues related to student/non-research personnel-supervisor collaboration. For 14 survey studies, a meta-analysis showed a pooled weighted average of 29% (95% CI 24% to 35%) researchers reporting their own or others' experience with misuse of authorship. Authorship misuse was reported more often by researcher outside of the USA and UK: 55% (95% CI 45% to 64%) for 4 studies in France, South Africa, India and Bangladesh vs. 23% (95% CI 18% to 28%) in USA/UK or international journal settings.High prevalence of authorship problems may have severe impact on the integrity of the research process, just as more serious forms of research misconduct. There is a need for more methodologically rigorous studies to understand the allocation of publication credit across research disciplines."
2160597895,"Open access, readership, citations: a randomized controlled trial of scientific journal publishing","Does free access to journal articles result in greater diffusion of scientific knowledge? Using a randomized controlled trial of open access publishing, involving 36 participating journals in the sciences, social sciences, and humanities, we report on the effects of free access on article downloads and citations. Articles placed in the open access condition (n=712) received significantly more downloads and reached a broader audience within the first year, yet were cited no more frequently, nor earlier, than subscription-access control articles (n=2533) within 3 yr. These results may be explained by social stratification, a process that concentrates scientific authors at a small number of elite research universities with excellent access to the scientific literature. The real beneficiaries of open access publishing may not be the research community but communities of practice that consume, but rarely contribute to, the corpus of literature."
2081305716,Characterizing Social Media Metrics of Scholarly Papers: The Effect of Document Properties and Collaboration Patterns,"A number of new metrics based on social media platforms—grouped under the term “altmetrics”—have recently been introduced as potential indicators of research impact. Despite their current popularity, there is a lack of information regarding the determinants of these metrics. Using publication and citation data from 1.3 million papers published in 2012 and covered in Thomson Reuters’ Web of Science as well as social media counts from Altmetric.com, this paper analyses the main patterns of five social media metrics as a function of document characteristics (i.e., discipline, document type, title length, number of pages and references) and collaborative practices and compares them to patterns known for citations. Results show that the presence of papers on social media is low, with 21.5% of papers receiving at least one tweet, 4.7% being shared on Facebook, 1.9% mentioned on blogs, 0.8% found on Google+ and 0.7% discussed in mainstream media. By contrast, 66.8% of papers have received at least one citation. Our findings show that both citations and social media metrics increase with the extent of collaboration and the length of the references list. On the other hand, while editorials and news items are seldom cited, it is these types of document that are the most popular on Twitter. Similarly, while longer papers typically attract more citations, an opposite trend is seen on social media platforms. Finally, contrary to what is observed for citations, it is papers in the Social Sciences and humanities that are the most often found on social media platforms. On the whole, these findings suggest that factors driving social media and citations are different. Therefore, social media metrics cannot actually be seen as alternatives to citations; at most, they may function as complements to other type of indicators."
2953237954,Science overlay maps: A new tool for research policy and library management,"Abstract We present a novel approach to visually locate bodies of research within the sciences, both at each moment of time and dynamically. This article describes how this approach fits with other efforts to locally and globally map scientific outputs. We then show how these science overlay maps help benchmarking, explore collaborations, and track temporal changes, using examples of universities, corporations, funding agencies, and research topics. We address their conditions of application and discuss advantages, downsides, and limitations. Overlay maps especially help investigate the increasing number of scientific developments and organizations that do not fit within traditional disciplinary categories. We make these tools available online to enable researchers to explore the ongoing sociocognitive transformations of science and technology systems."
2053928690,Using the h‐index to rank influential information scientistss,"Abstract The authors apply a new bibliometric measure, the h ‐index (Hirsch, 2005), to the literature of information science. Faculty rankings based on raw citation counts are compared with those based on h ‐counts. There is a strong positive correlation between the two sets of rankings. It is shown how the h ‐index can be used to express the broad impact of a scholar's research output over time in more nuanced fashion than straight citation counts."
2096123611,"Requirements for a cocitation similarity measure, with special reference to Pearson's correlation coefficient","Abstract Author cocitation analysis (ACA), a special type of cocitation analysis, was introduced by White and Griffith in 1981. This technique is used to analyze the intellectual structure of a given scientific field. In 1990, McCain published a technical overview that has been largely adopted as a standard. Here, McCain notes that Pearson's correlation coefficient (Pearson's r ) is often used as a similarity measure in ACA and presents some advantages of its use. The present article criticizes the use of Pearson's r in ACA and sets forth two natural requirements that a similarity measure applied in ACA should satisfy. It is shown that Pearson's r does not satisfy these requirements. Real and hypothetical data are used in order to obtain counterexamples to both requirements. It is concluded that Pearson's r is probably not an optimal choice of a similarity measure in ACA. Still, further empirical research is needed to show if, and in that case to what extent, the use of similarity measures in ACA that fulfill these requirements would lead to objectively better results in full‐scale studies. Further, problems related to incomplete cocitation matrices are discussed."
1931854307,Ten-Year Cross-Disciplinary Comparison of the Growth of Open Access and How it Increases Research Citation Impact,"Lawrence (2001)found computer science articles that were openly accessible (OA) on the Web were cited more. We replicated this in physics. We tested 1,307,038 articles published across 12 years (1992-2003) in 10 disciplines (Biology, Psychology, Sociology, Health, Political Science, Economics, Education, Law, Business, Management). A robot trawls the Web for full-texts using reference metadata ISI citation data (signal detectability d'=2.45; bias = 0.52). Percentage OA (relative to total OA + NOA) articles varies from 5%-16% (depending on discipline, year and country) and is slowly climbing annually (correlation r=.76, sample size N=12, probability p  .90, N=12, p < .0005) and the effect is greater with the more highly cited articles (r = .98, N=6, p < .005). Causality cannot be determined from these data, but our prior finding of a similar pattern in physics, where percent OA is much higher (and even approaches 100% in some subfields), makes it unlikely that the OA citation advantage is merely or mostly a self-selection bias (for making only one's better articles OA). Further research will analyze the effect's timing, causal components and relation to other variables."
2039027543,Bibliometric Methods: Pitfalls and Possibilities,"Bibliometric studies are increasingly being used for research assessment. Bibliometric indicators are strongly methodology-dependent but for all of them, various types of data normalization are an indispensable requirement. Bibliometric studies have many pitfalls; technical skill, critical sense and a precise knowledge about the examined scientific domain are required to carry out and interpret bibliometric investigations correctly."
2115339903,Anatomy of open access publishing: a study of longitudinal development and internal structure,"Open access (OA) is a revolutionary way of providing access to the scholarly journal literature made possible by the Internet. The primary aim of this study was to measure the volume of scientific articles published in full immediate OA journals from 2000 to 2011, while observing longitudinal internal shifts in the structure of OA publishing concerning revenue models, publisher types and relative distribution among scientific disciplines. The secondary aim was to measure the share of OA articles of all journal articles, including articles made OA by publishers with a delay and individual author-paid OA articles in subscription journals (hybrid OA), as these subsets of OA publishing have mostly been ignored in previous studies.Stratified random sampling of journals in the Directory of Open Access Journals (n = 787) was performed. The annual publication volumes spanning 2000 to 2011 were retrieved from major publication indexes and through manual data collection.An estimated 340,000 articles were published by 6,713 full immediate OA journals during 2011. OA journals requiring article-processing charges have become increasingly common, publishing 166,700 articles in 2011 (49% of all OA articles). This growth is related to the growth of commercial publishers, who, despite only a marginal presence a decade ago, have grown to become key actors on the OA scene, responsible for 120,000 of the articles published in 2011. Publication volume has grown within all major scientific disciplines, however, biomedicine has seen a particularly rapid 16-fold growth between 2000 (7,400 articles) and 2011 (120,900 articles). Over the past decade, OA journal publishing has steadily increased its relative share of all scholarly journal articles by about 1% annually. Approximately 17% of the 1.66 million articles published during 2011 and indexed in the most comprehensive article-level index of scholarly articles (Scopus) are available OA through journal publishers, most articles immediately (12%) but some within 12 months of publication (5%).OA journal publishing is disrupting the dominant subscription-based model of scientific publishing, having rapidly grown in relative annual share of published journal articles during the last decade."
2737988770,A multi-disciplinary perspective on emergent and future innovations in peer review,"Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure? Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research. With the advent of web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review. We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation. Ideally, any new systems will demonstrate that they out-perform and reduce the biases of existing models as much as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages. We also propose a novel hybrid platform model that could, at least partially, resolve many of the socio-technical issues associated with peer review, and potentially disrupt the entire scholarly communication system. Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments."
4232875366,Open Access,"A concise introduction to the basics of open access, describing what it is (and isn't) and showing that it is easy, fast, inexpensive, legal, and beneficial. The Internet lets us share perfect copies of our work with a worldwide audience at virtually no cost. We take advantage of this revolutionary opportunity when we make our work “open access”: digital, online, free of charge, and free of most copyright and licensing restrictions. Open access is made possible by the Internet and copyright-holder consent, and many authors, musicians, filmmakers, and other creators who depend on royalties are understandably unwilling to give their consent. But for 350 years, scholars have written peer-reviewed journal articles for impact, not for money, and are free to consent to open access without losing revenue. In this concise introduction, Peter Suber tells us what open access is and isn't, how it benefits authors and readers of research, how we pay for it, how it avoids copyright problems, how it has moved from the periphery to the mainstream, and what its future may hold. Distilling a decade of Suber's influential writing and thinking about open access, this is the indispensable book on the subject for researchers, librarians, administrators, funders, publishers, and policy makers."
2163539724,Bibliometric Indicators: Quality Measurements of Scientific Publication,"Bibliometrics is a set of mathematical and statistical methods used to analyze and measure the quantity and quality of books, articles, and other forms of publications. There are three types of bibliometric indicators: quantity indicators, which measure the productivity of a particular researcher; quality indicators, which measure the quality (or ""performance"") of a researcher's output; and structural indicators, which measure connections between publications, authors, and areas of research. Bibliometric indicators are especially important for researchers and organizations, as these measurements are often used in funding decisions, appointments, and promotions of researchers. As more and more scientific discoveries occur and published research results are read and then quoted by other researchers, bibliometric indicators are becoming increasingly important. This article provides an overview of the currently used bibliometric indicators and summarizes the critical elements and characteristics one should be aware of when evaluating the quantity and quality of scientific output."
2137892569,Author Sequence and Credit for Contributions in Multiauthored Publications,"The increasing tendency across scientific disciplines to write multiauthored papers [1,2] makes the issue of the sequence of contributors' names a major topic both in terms of reflecting actual contributions and in a posteriori assessments by evaluation committees. Traditionally, the first author contributes most and also receives most of the credit, whereas the position of subsequent authors is usually decided by contribution, alphabetical order, or reverse seniority. Ranking the first or second author in a two-author paper is straightforward, but the meaning of position becomes increasingly arbitrary as the number of authors increases beyond two. Criteria for authorship have been discussed at length, because of the inflationary increase in the number of authors on papers submitted to biomedical journals and the practice of “gift” authorship [3,4], but a simple way to determine credit associated with the sequence of authors' names is still missing [4–7] (http://www.councilscienceeditors.org).The situation in our area of research—the ecological and environmental sciences—has changed in recent years. Following informal practices in the biomedical sciences, the last author often gets as much credit as the first author, because he or she is assumed to be the driving force, both intellectually and financially, behind the research. Evaluation committees and funding bodies often take last authorship as a sign of successful group leadership and make this a criterion in hiring, granting, and promotion. This practice is unofficial, and hence not always followed, meaning that sometimes last authors “mistakenly” benefit when they actually are not principal investigators. Moreover, there is no accepted yardstick in assessing the actual contribution of a group leader to given scientific publications [8,9], so interpretation of author sequence can be like a lottery. Hence, one really does not know if being last author means that the overall contribution was the most or least important.Although reducing evaluation of authors' complex contributions to simple metrics is regrettable, in reality it is already in practice in most evaluation committees. Hence, in our opinion, we need a simple and straightforward approach to estimate the credit associated with the sequence of authors' names that is free from any arbitrary rank valuation. In multiauthored papers, the first author position should clearly be assigned to the individual making the greatest contribution [4–6], as is common practice. However, authors often adopt different methods of crediting contributions for the following authors, because of very different traditions across countries and research fields, resulting in very different criteria that committees adopt to quantify author's contributions [8,9]. For example, some authors use alphabetical sequence, while others think that the last author position has great importance or that the second author position is the second most important. Still others detail each author's contribution in a footnote.We suggest that the approach taken should be stated in the acknowledgements section, and evaluation committees are asked to weigh the contribution of each author based on the criteria given by the authors. This would make reviewers aware that there are different cultures to authorship order. The usual and informal practice of giving the whole credit (impact factor) to each author of a multiauthored paper is not adequate and overemphasises the minor contributions of many authors (Table 1). Similarly, evaluation of authors according to citation frequencies means often overrating resulting from high-impact but multiauthored publications. The following approaches may be identified.Table 1Comparison of the Credit for Contributions to This Paper under the Four Different Models Suggested in the Text(1) The “sequence-determines-credit” approach (SDC). The sequence of authors should reflect the declining importance of their contribution, as suggested b"
1531607676,Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows,"Although there is evidence that counting the readers of an article in the social reference site, M endeley, may help to capture its research impact, the extent to which this is true for different scientific fields is unknown. In this study, we compare M endeley readership counts with citations for different social sciences and humanities disciplines. The overall correlation between M endeley readership counts and citations for the social sciences was higher than for the humanities. Low and medium correlations between M endeley bookmarks and citation counts in all the investigated disciplines suggest that these measures reflect different aspects of research impact. M endeley data were also used to discover patterns of information flow between scientific fields. Comparing information flows based on M endeley bookmarking data and cross‐disciplinary citation analysis for the disciplines revealed substantial similarities and some differences. Thus, the evidence from this study suggests that M endeley readership data could be used to help capture knowledge transfer across scientific disciplines, especially for people that read but do not author articles, as well as giving impact evidence at an earlier stage than is possible with citation counts."
2768281947,Reviewer bias in single- versus double-blind peer review,"Significance Scientific peer review has been a cornerstone of the scientific method since the 1600s. Debate continues regarding the merits of single-blind review, in which anonymous reviewers know the authors of a paper and their affiliations, compared with double-blind review, in which this information is hidden. We present an experimental study of this question. In computer science, research often appears first or exclusively in peer-reviewed conferences rather than journals. Our study considers full-length submissions to the highly selective 2017 Web Search and Data Mining conference (15.6% acceptance rate). Each submission is simultaneously scored by two single-blind and two double-blind reviewers. Our analysis shows that single-blind reviewing confers a significant advantage to papers with famous authors and authors from high-prestige institutions."
2171817276,Defining and identifying Sleeping Beauties in science,"A Sleeping Beauty (SB) in science refers to a paper whose importance is not recognized for several years after publication. Its citation history exhibits a long hibernation period followed by a sudden spike of popularity. Previous studies suggest a relative scarcity of SBs. The reliability of this conclusion is, however, heavily dependent on identification methods based on arbitrary threshold parameters for sleeping time and number of citations, applied to small or monodisciplinary bibliographic datasets. Here we present a systematic, large-scale, and multidisciplinary analysis of the SB phenomenon in science. We introduce a parameter-free measure that quantifies the extent to which a specific paper can be considered an SB. We apply our method to 22 million scientific papers published in all disciplines of natural and social sciences over a time span longer than a century. Our results reveal that the SB phenomenon is not exceptional. There is a continuous spectrum of delayed recognition where both the hibernation period and the awakening intensity are taken into account. Although many cases of SBs can be identified by looking at monodisciplinary bibliographic data, the SB phenomenon becomes much more apparent with the analysis of multidisciplinary datasets, where we can observe many examples of papers achieving delayed yet exceptional importance in disciplines different from those where they were originally published. Our analysis emphasizes a complex feature of citation dynamics that so far has received little attention, and also provides empirical evidence against the use of short-term citation metrics in the quantification of scientific impact."
1981886524,SciMAT: A new science mapping analysis software tool,"This article presents a new open‐source software tool, SciMAT , which performs science mapping analysis within a longitudinal framework. It provides different modules that help the analyst to carry out all the steps of the science mapping workflow. In addition, SciMAT presents three key features that are remarkable in respect to other science mapping software tools: (a) a powerful preprocessing module to clean the raw bibliographical data, (b) the use of bibliometric measures to study the impact of each studied element, and (c) a wizard to configure the analysis."
1553564559,Altmetrics in the wild: Using social media to explore scholarly impact,"In growing numbers, scholars are integrating social media tools like blogs, Twitter, and Mendeley into their professional communications. The online, public nature of these tools exposes and reifies scholarly processes once hidden and ephemeral. Metrics based on this activities could inform broader, faster measures of impact, complementing traditional citation metrics. This study explores the properties of these social media-based metrics or ""altmetrics"", sampling 24,331 articles published by the Public Library of Science. We find that that different indicators vary greatly in activity. Around 5% of sampled articles are cited in Wikipedia, while close to 80% have been included in at least one Mendeley library. There is, however, an encouraging diversity; a quarter of articles have nonzero data from five or more different sources. Correlation and factor analysis suggest citation and altmetrics indicators track related but distinct impacts, with neither able to describe the complete picture of scholarly use alone. There are moderate correlations between Mendeley and Web of Science citation, but many altmetric indicators seem to measure impact mostly orthogonal to citation. Articles cluster in ways that suggest five different impact ""flavors"", capturing impacts of different types on different audiences; for instance, some articles may be heavily read and saved by scholars but seldom cited. Together, these findings encourage more research into altmetrics as complements to traditional citation measures."
2118839838,Turning the tables on citation analysis one more time: Principles for comparing sets of documents,"We submit newly developed citation impact indicators based not on arithmetic averages of citations but on percentile ranks. Citation distributions are—as a rule—highly skewed and should not be arithmetically averaged. With percentile ranks, the citation score of each paper is rated in terms of its percentile in the citation distribution. The percentile ranks approach allows for the formulation of a more abstract indicator scheme that can be used to organize and/or schematize different impact indicators according to three degrees of freedom: the selection of the reference sets, the evaluation criteria, and the choice of whether or not to define the publication sets as independent. Bibliometric data of seven principal investigators (PIs) of the Academic Medical Center of the University of Amsterdam are used as an exemplary dataset. We demonstrate that the proposed family indicators [R(6), R(100), R(6, k), R(100, k)] are an improvement on averages-based indicators because one can account for the shape of the distributions of citations over papers."
2146697111,Honorary and ghost authorship in high impact biomedical journals: a cross sectional survey,"To assess the prevalence of honorary and ghost authors in six leading general medical journals in 2008 and compare this with the prevalence reported by authors of articles published in 1996.Cross sectional survey using a web based questionnaire.International survey of journal authors.Sample of corresponding authors of 896 research articles, review articles, and editorial/opinion articles published in six general medical journals with high impact factors in 2008: Annals of Internal Medicine, JAMA, Lancet, Nature Medicine, New England Journal of Medicine, and PLoS Medicine.Self reported compliance with International Committee of Medical Journal Editors (ICMJE) criteria for authorship for all authors on the selected articles.A total of 630/896 (70.3%) corresponding authors responded to the survey. The prevalence of articles with honorary authorship or ghost authorship, or both, was 21.0% (95% CI 18.0% to 24.3%), a decrease from 29.2% reported in 1996 (P = 0.004). Based on 545 responses on honorary authorship, 96 articles (17.6% (95% CI 14.6% to 21.0%)) had honorary authors (range by journal 12.2% to 29.3%), a non-significant change from 1996 (19.3%; P = 0.439). Based on 622 responses on ghost authorship, 49 articles (7.9% (6.0% to 10.3%)) had ghost authors (range by journal 2.1% to 11.0%), a significant decline from 1996 (11.5%; P = 0.023). The prevalence of honorary authorship was 25.0% in original research reports, 15.0% in reviews, and 11.2% in editorials, whereas the prevalence of ghost authorship was 11.9% in research articles, 6.0% in reviews, and 5.3% in editorials.Evidence of honorary and ghost authorship in 21% of articles published in major medical journals in 2008 suggests that increased efforts by scientific journals, individual authors, and academic institutions are essential to promote responsibility, accountability, and transparency in authorship, and to maintain integrity in scientific publication."
2103948822,Deep impact: unintended consequences of journal rank,"Most researchers acknowledge an intrinsic hierarchy in the scholarly journals (""journal rank"") that they submit their work to, and adjust not only their submission but also their reading strategies accordingly. On the other hand, much has been written about the negative effects of institutionalizing journal rank as an impact measure. So far, contributions to the debate concerning the limitations of journal rank as a scientific impact assessment tool have either lacked data, or relied on only a few studies. In this review, we present the most recent and pertinent data on the consequences of our current scholarly communication system with respect to various measures of scientific quality (such as utility/citations, methodological soundness, expert ratings or retractions). These data corroborate previous hypotheses: using journal rank as an assessment tool is bad scientific practice. Moreover, the data lead us to argue that any journal rank (not only the currently-favored Impact Factor) would have this negative impact. Therefore, we suggest that abandoning journals altogether, in favor of a library-based scholarly communication system, will ultimately be necessary. This new system will use modern information technology to vastly improve the filter, sort and discovery functions of the current journal system."
1975947821,Who publishes in “predatory” journals?,"Many open access journals have a reputation for being of low quality and being dishonest with regard to peer review and publishing costs. Such journals are labeled “predatory” journals. This study examines author profiles for some of these “predatory” journals as well as for groups of more well‐recognized open access journals. We collect and analyze the publication record, citation count, and geographic location of authors from the various groups of journals. Statistical analyses verify that each group of journals has a distinct author population. Those who publish in “predatory” journals are, for the most part, young and inexperienced researchers from developing countries. We believe that economic and sociocultural conditions in these developing countries have contributed to the differences found in authorship between “predatory” and “nonpredatory” journals."
2129286173,Show me the data,"![Graphic][1] ©cartoonbank.com. All Rights Reserved. The integrity of data, and transparency about their acquisition, are vital to science. The impact factor data that are gathered and sold by Thomson Scientific (formerly the Institute of Scientific Information, or ISI) have a strong"
2097509366,A study of open access journals using article processing charges,"Article processing charges ( APC s) are a central mechanism for funding open access (OA) scholarly publishing. We studied the APCs charged and article volumes of journals that were listed in the Directory of Open Access Journals as charging APCs. These included 1,370 journals that published 100,697 articles in 2010. The average APC was $906 U.S. dollars (USD) calculated over journals and $904 USD calculated over articles. The price range varied between $8 and $3,900 USD, with the lowest prices charged by journals published in developing countries and the highest by journals with high‐impact factors from major international publishers. Journals in biomedicine represent 59% of the sample and 58% of the total article volume. They also had the highest APCs of any discipline. Professionally published journals, both for profit and nonprofit, had substantially higher APCs than journals published by societies, universities, or scholars/researchers. These price estimates are lower than some previous studies of OA publishing and much lower than is generally charged by subscription publishers making individual articles OA in what are termed hybrid journals."
1837245417,UNDERSTANDING THE LIMITATIONS OF THE JOURNAL IMPACT FACTOR,"The impact factor, a simple mathematical formula reflecting the number of citations of a journal's material divided by the number of citable materials published by that same journal, has evolved to become one of the most influential tools in modern research and academia. The impact factor can be influenced and biased (intentionally or otherwise) by many factors. Extension of the impact factor to the assessment of journal quality or individual authors is inappropriate. Extension of the impact factor to cross-discipline journal comparison is also inappropriate. Those who choose to use the impact factor as a comparative tool should be aware of the nature and premise of its derivation and also of its inherent flaws and practical limitations."
3121419210,How Much Better Are the Most-Prestigious Journals? The Statistics of Academic Publication,"Articles in high-prestige journals receive more citations and more applause than articles in less-prestigious journals, but how much more do these articles contribute to knowledge? This article uses a statistical theory of review processes to draw inferences about differences value between articles in more-prestigious versus less-prestigious journals. This analysis indicates that there is much overlap in articles in different prestige strata. Indeed, theory implies that about half of the articles published are not among the best ones submitted to those journals, and some of the manuscripts that belong in the highest-value 20% have the misfortune to elicit rejections from as many as five journals. Some social science departments and business schools strongly emphasize publication in prestigious journals. Although one can draw inferences about an author’s average manuscript from the percentage in top-tier journals, the confidence limits for such inferences are wide. A focus on prestigious journals may benefit the most prestigious departments or schools but add randomness to the decisions of departments or schools that are not at the very top. Such a focus may also impede the development of knowledge when mediocre research receives the endorsement of high visibility."
2150443611,Team Assembly Mechanisms Determine Collaboration Network Structure and Team Performance,"Agents in creative enterprises are embedded in networks that inspire, support, and evaluate their work. Here, we investigate how the mechanisms by which creative teams self-assemble determine the structure of these collaboration networks. We propose a model for the self-assembly of creative teams that has its basis in three parameters: team size, the fraction of newcomers in new productions, and the tendency of incumbents to repeat previous collaborations. The model suggests that the emergence of a large connected community of practitioners can be described as a phase transition. We find that team assembly mechanisms determine both the structure of the collaboration network and team performance for teams derived from both artistic and scientific fields."
1550064549,The access principle: the case for open access to research and scholarship,"product of a highly talented team of undergraduates and graduate students who have come together in the Public Knowledge Project at the University of British Columbia.The principal piece of software, known as Open Journal Systems, has contributed much to my understanding of online journal processes, economics, indexing, and reading, and figures as such in this book.Additionally, this effort to build robust software that improves the quality of access to journals proved an excellent focal point for discussing the possibilities of open access publishing with researchers, editors, librarians, and publishers in many parts of the world.Open Journal Systems has turned out to be more than a talking point and a test bed for the ideas discussed in this book.It has moved beyond the proof-of-concept stage, with the assistance and encouragement of an international open source community, and is now being used to publish open access journals, as well as some subscription journals around the world, with versions now available in seven languages.Given the interest shown in this open source software, a partnership was formed in early 2005 among the Public Knowledge Project, the Canadian Center for Studies in Publishing at Simon Fraser University, and Simon Fraser University Library to oversee the long-term development of Open Journal Systems, Open Conference Systems, and the PKP Harvester.This book, however, is not about the development of publishing software; it is about the age-old question of access to knowledge.In considering what open access has to offer on that question in this book, this is a work of inquiry and advocacy.Its goal is to inform and inspire a larger debate over the political and moral economy of knowledge that will constitute the future of research.It seeks to elevate the questions currently being raised about how research is published, so that they are seen to shine a greater light on our work as scholars and as citizens of a larger world.And at this historic moment, in this transition in journal publishing from print to digital formats, the model of open access publishing challenges not only traditional methods of publishing scholarly work, but the very presence and place of this knowledge in the world.What, then, of the all-too-obvious irony of publishing a book in print and on sale in bookstores about making online research free for the world?I have published and circulated earlier versions of most of these"
3006753318,Historical comparison of gender inequality in scientific careers across countries and disciplines,"There is extensive, yet fragmented, evidence of gender differences in academia suggesting that women are underrepresented in most scientific disciplines and publish fewer articles throughout a career, and their work acquires fewer citations. Here, we offer a comprehensive picture of longitudinal gender differences in performance through a bibliometric analysis of academic publishing careers by reconstructing the complete publication history of over 1.5 million gender-identified authors whose publishing career ended between 1955 and 2010, covering 83 countries and 13 disciplines. We find that, paradoxically, the increase of participation of women in science over the past 60 years was accompanied by an increase of gender differences in both productivity and impact. Most surprisingly, though, we uncover two gender invariants, finding that men and women publish at a comparable annual rate and have equivalent career-wise impact for the same size body of work. Finally, we demonstrate that differences in publishing career lengths and dropout rates explain a large portion of the reported career-wise differences in productivity and impact, although productivity differences still remain. This comprehensive picture of gender inequality in academia can help rephrase the conversation around the sustainability of women's careers in academia, with important consequences for institutions and policy makers."
1536128219,The Effects of Double-Blind versus Single-Blind Reviewing: Experimental Evidence from The American Economic Review,"The results from a randomized experiment conducted at The American Economic Review on the effects of double-blind versus single-blind peer reviewing on acceptance rates and referee ratings indicate that acceptance rates are lower and referees are more critical when the reviewer is unaware of the author's identity. These patterns are not significantly different between female and male authors. Authors at top-ranked universities and at colleges and low-ranked universities are largely unaffected by the different reviewing practices, but authors at near-top-ranked universities and at nonacademic institutions have lower acceptance rates under double-blind reviewing. (JEL A12, C93) Scholars have long been divided regarding the advantages and disadvantages of single-blind versus double-blind reviewing in their scholarly journals. Among 38 wellknown journals in chemistry, biology, physics, mathematics, history, psychology, political science, sociology, and anthropology, 11 send submitted papers for outside peer review with the, author's name and affiliation removed from the front of the paper (double-blind), while the remainder send submitted papers to referees with"
3098217728,The structure and dynamics of cocitation clusters: A multiple‐perspective cocitation analysis,"A multiple-perspective co-citation analysis method is introduced for characterizing and interpreting the structure and dynamics of co-citation clusters. The method facilitates analytic and sense making tasks by integrating network visualization, spectral clustering, automatic cluster labeling, and text summarization. Co-citation networks are decomposed into co-citation clusters. The interpretation of these clusters is augmented by automatic cluster labeling and summarization. The method focuses on the interrelations between a co-citation cluster's members and their citers. The generic method is applied to a three-part analysis of the field of Information Science as defined by 12 journals published between 1996 and 2008: 1) a comparative author co-citation analysis (ACA), 2) a progressive ACA of a time series of co-citation networks, and 3) a progressive document co-citation analysis (DCA). Results show that the multiple-perspective method increases the interpretability and accountability of both ACA and DCA networks."
1971648300,Peer review in a changing world: An international study measuring the attitudes of researchers,"This large‐scale international study measures the attitudes of more than 4,000 researchers toward peer review. In 2009, 40,000 authors of research papers from across the globe were invited to complete an online survey. Researchers were asked to rate a number of general statements about peer review, and then a subset of respondents, who had themselves peer reviewed, rated a series of statements concerning their experience of peer review. The study found that the peer review process is highly regarded by the vast majority of researchers and considered by most to be essential to the communication of scholarly research. Nine out of 10 authors believe that peer review improved the last paper they published. Double‐blind peer review is considered the most effective form of peer review. Nearly three quarters of researchers think that technological advances are making peer review more effective. Most researchers believe that although peer review should identify fraud, it is very difficult for it to do so. Reviewers are committed to conducting peer review in the future and believe that simple practical steps, such as training new reviewers would further improve peer review."
2013811657,Impact Factor Distortions,"This Editorial coincides with the release of the San Francisco declaration on research Assessment (DORA), the outcome of a gathering of concerned scientists at the December 2012 meeting of the American Society for Cell Biology. * To correct distortions in the evaluation of scientific research, DORA aims to stop the use of the ""journal impact factor"" in judging an individual scientist's work. The Declaration states that the impact factor must not be used as ""a surrogate measure of the quality of individual research articles, to assess an individual scientist's contributions, or in hiring, promotion, or funding decisions."" DORA also provides a list of specific actions, targeted at improving the way scientific publications are assessed, to be taken by funding agencies, institutions, publishers, researchers, and the organizations that supply metrics. These recommendations have thus far been endorsed by more than 150 leading scientists and 75 scientific organizations, including the American Association for the Advancement of Science (the publisher of Science ). Here are some reasons why:"
2952984634,Comparing bibliometric statistics obtained from the Web of Science and Scopus,"Abstract For more than 40 years, the Institute for Scientific Information (ISI, now part of Thomson Reuters) produced the only available bibliographic databases from which bibliometricians could compile large‐scale bibliometric indicators. ISI's citation indexes, now regrouped under the Web of Science (WoS), were the major sources of bibliometric data until 2004, when Scopus was launched by the publisher Reed Elsevier. For those who perform bibliometric analyses and comparisons of countries or institutions, the existence of these two major databases raises the important question of the comparability and stability of statistics obtained from different data sources. This paper uses macrolevel bibliometric indicators to compare results obtained from the WoS and Scopus. It shows that the correlations between the measures obtained with both databases for the number of papers and the number of citations received by countries, as well as for their ranks, are extremely high (R 2 ≈ .99). There is also a very high correlation when countries' papers are broken down by field. The paper thus provides evidence that indicators of scientific production and citations at the country level are stable and largely independent of the database."
4254361288,Intellectual Collaboration,"Intellectual collaboration in science includes formal coauthorship as well as presentation of papers at workshops, seminars, and professional meetings and informal commentary from colleagues, journal referees, and editors. While the incidence and extent of formal coauthorship are greater in biology than in economics, the extent of intellectual collaboration is greater in economics than in biology. Intellectual property rights to coauthored papers in economics tend to be assigned alphabetically, whereas biology is characterized by a strong merit‐based (nonalphabetical) assignment of intellectual property rights. These patterns do not result from differences in the relative importance of funding/physical capital."
2151568674,Detecting h-index manipulation through self-citation analysis,"The h-index has received an enormous attention for being an indicator that measures the quality of researchers and organizations. We investigate to what degree authors can inflate their h-index through strategic self-citations with the help of a simulation. We extended Burrell's publication model with a procedure for placing self-citations, following three different strategies: random self-citation, recent self-citations and h-manipulating self-citations. The results show that authors can considerably inflate their h-index through self-citations. We propose the q-index as an indicator for how strategically an author has placed self-citations, and which serves as a tool to detect possible manipulation of the h-index. The results also show that the best strategy for an high h-index is publishing papers that are highly cited by others. The productivity has also a positive effect on the h-index."
1514934734,Multidimensional Journal Evaluation: Analyzing Scientific Periodicals beyond the Impact Factor,"Sirje Virkus (Tallinn,"
1559698545,Coauthorship patterns and trends in the sciences (1980-1998): A bibliometric study with implications for database indexing and search strategies,"The present study aims at describing both the common and the distinguishing features of coauthorship trends and patterns in selected science fields. The relation between coauthorship schemes and other bibliometric features, such as publication activity and citation impact are analyzed. I show that, while copublication activity has grown considerably, the extent of coauthorship and its relation with productivity and citation impact largely varies among fields. Besides universally valid tendencies, subject specific features can be found."
2074975096,Collaborative Research Across Disciplinary and Organizational Boundaries,"Scientific and engineering research increasingly involves multidisciplinary collaboration, sometimes across multiple organizations. Technological advances have made such cross-boundary projects possible, yet they can carry high coordination costs. This study investigated scientific collaboration across disciplinary and university boundaries to understand the need for coordination in these collaborations and how different levels of coordination predicted success. We conducted a study of 62 scientific collaborations supported by a program of the US National Science Foundation in 1998 and 1999. Projects with principal investigators (PIs) in more disciplines reported as many positive outcomes as did projects involving fewer disciplines. By contrast, multi-university, rather than multidisciplinary, projects were problematic. Projects with PIs from more universities were significantly less well coordinated and reported fewer positive outcomes than projects with PIs from fewer universities. Coordination mechanisms that brought distant researchers together physically slightly reduced the negative impact of collaborations involving multiple universities. We discuss implications for theory, practice, and policy."
1932742904,An Overview of Microsoft Academic Service (MAS) and Applications,"In this paper we describe a new release of a Web scale entity graph that serves as the backbone of Microsoft Academic Service (MAS), a major production effort with a broadened scope to the namesake vertical search engine that has been publicly available since 2008 as a research prototype. At the core of MAS is a heterogeneous entity graph comprised of six types of entities that model the scholarly activities: field of study, author, institution, paper, venue, and event. In addition to obtaining these entities from the publisher feeds as in the previous effort, we in this version include data mining results from the Web index and an in-house knowledge base from Bing, a major commercial search engine. As a result of the Bing integration, the new MAS graph sees significant increase in size, with fresh information streaming in automatically following their discoveries by the search engine. In addition, the rich entity relations included in the knowledge base provide additional signals to disambiguate and enrich the entities within and beyond the academic domain. The number of papers indexed by MAS, for instance, has grown from low tens of millions to 83 million while maintaining an above 95% accuracy based on test data sets derived from academic activities at Microsoft Research. Based on the data set, we demonstrate two scenarios in this work: a knowledge driven, highly interactive dialog that seamlessly combines reactive search and proactive suggestion experience, and a proactive heterogeneous entity recommendation."
2156577392,Towards a new crown indicator: an empirical analysis,"We present an empirical comparison between two normalization mechanisms for citation-based indicators of research performance. These mechanisms aim to normalize citation counts for the field and the year in which a publication was published. One mechanism is applied in the current so-called crown indicator of our institute. The other mechanism is applied in the new crown indicator that our institute is currently exploring. We find that at high aggregation levels, such as at the level of large research institutions or at the level of countries, the differences between the two mechanisms are very small. At lower aggregation levels, such as at the level of research groups or at the level of journals, the differences between the two mechanisms are somewhat larger. We pay special attention to the way in which recent publications are handled. These publications typically have very low citation counts and should therefore be handled with special care."
1983454294,Citation Statistics from 110 Years of Physical Review,Publicly available data reveal long-term systematic features about citation statistics and how papers are referenced. The data also tell fascinating citation histories of individual articles.
2021082438,A cast of thousands: Coauthorship and subauthorship collaboration in the 20th century as manifested in the scholarly journal literature of psychology and philosophy,"Abstract We chronicle the use of acknowledgments in 20th‐century scholarship by analyzing and classifying more than 4,500 specimens covering a 100‐year period. Our results show that the intensity of acknowledgment varies by discipline, reflecting differences in prevailing sociocognitive structures and work practices. We demonstrate that the acknowledgment has gradually established itself as a constitutive element of academic writing, one that provides a revealing insight into the nature and extent of subauthorship collaboration. Complementary data on rates of coauthorship are also presented to highlight the growing importance of collaboration and the increasing division of labor in contemporary research and scholarship."
2135696593,Citation indexes for science. A new dimension in documentation through association of ideas†,"‘The uncritical citation of disputed data by a writer, whether it be deliberate or not, is a serious matter. Of course, knowingly propagandizing unsubstantiated claims is particularly abhorrent, but just as many naive students may be swayed by unfounded assertions presented by a writer who is unaware of the criticisms. Buried in scholarly journals, critical notes are increasingly likely to be overlooked with the passage of time, while the studies to which they pertain, having been reported more widely, are apt to be rediscovered.’ 1"
2132615533,The place of serials in referencing practices: Comparing natural sciences and engineering with social sciences and humanities,"Abstract Journal articles constitute the core documents for the diffusion of knowledge in the natural sciences. It has been argued that the same is not true for the social sciences and humanities where knowledge is more often disseminated in monographs that are not indexed in the journal‐based databases used for bibliometric analysis. Previous studies have made only partial assessments of the role played by both serials and other types of literature. The importance of journal literature in the various scientific fields has therefore not been systematically characterized. The authors address this issue by providing a systematic measurement of the role played by journal literature in the building of knowledge in both the natural sciences and engineering and the social sciences and humanities. Using citation data from the CD‐ROM versions of the Science Citation Index (SCI), Social Science Citation Index (SSCI), and Arts and Humanities Citation Index (AHCI) databases from 1981 to 2000 (Thomson ISI, Philadelphia, PA), the authors quantify the share of citations to both serials and other types of literature. Variations in time and between fields are also analyzed. The results show that journal literature is increasingly important in the natural and social sciences, but that its role in the humanities is stagnant and has even tended to diminish slightly in the 1990s. Journal literature accounts for less than 50% of the citations in several disciplines of the social sciences and humanities; hence, special care should be used when using bibliometric indicators that rely only on journal literature."
2095126174,100 Most Cited Articles in Orthopaedic Surgery,"Citation analysis reflects the recognition a work has received in the scientific community by its peers, and is a common method to determine 'classic' works in medical specialties.We determined which published articles in orthopaedic journals have been most cited by other authors by ranking the 100 top-cited works. By analyzing characteristics of these articles, we intended to determine what qualities make an orthopaedic article important to the specialty. Finally, we determined if there was a change in level of evidence of studies on this list with time.Science Citation Index Expanded was searched for citations to articles published in any of the 49 journals in the subject category ""ORTHOPEDICS."" Each of the 49 journals was searched separately using the ""cited reference search"" to determine the 100 most often cited articles. Each article was reviewed for basic information including year of publication, country of origin, source journal of the article, article type, and level of evidence. We categorized the journal article by field of research where possible.The number of citations ranged from 1748 to 353. The 100 most often cited articles in orthopaedic surgery were published in 11 of the 49 journals, spanning from general to more specific subspecialty journals. The majority of the papers (76) were clinical, with the remaining representing some type of basic science research. The most common level of evidence was IV (42 of the 76 studies). Of the 76 clinical articles, 27 introduced or tested classification systems or outcome measurement tools.Authors aiming to write a highly cited article in an orthopaedic surgery journal will be favored by language of publication, source journal, country of origin, and introduction of a classification scheme or outcome tool."
2088336913,Automatic classification of citation function,"Citation function is defined as the author's reason for citing a given paper (e.g. acknowledgement of the use of the cited method). The automatic recognition of the rhetorical function of citations in scientific text has many applications, from improvement of impact factor calculations to text summarisation and more informative citation indexers. We show that our annotation scheme for citation function is reliable, and present a supervised machine learning framework to automatically classify citation function, using both shallow and linguistically-inspired features. We find, amongst other things, a strong relationship between citation function and sentiment classification."
1552729918,Open access self-archiving: An author study,"This, our second author international, cross-disciplinary study on open access had 1296 respondents. Its focus was on self-archiving. Almost half (49%) of the respondent population have self-archived at least one article during the last three years. Use of institutional repositories for this purpose has doubled and usage has increased by almost 60% for subject-based repositories. Self-archiving activity is greatest amongst those who publish the largest number of papers. There is still a substantial proportion of authors unaware of the possibility of providing open access to their work by self-archiving. Of the authors who have not yet self-archived any articles, 71% remain unaware of the option. With 49% of the author population having self-archived in some way, this means that 36% of the total author population (71% of the remaining 51%), has not yet been appraised of this way of providing open access. Authors have frequently expressed reluctance to self-archive because of the perceived time required and possible technical difficulties in carrying out this activity, yet findings here show that only 20% of authors found some degree of difficulty with the first act of depositing an article in a repository, and that this dropped to 9% for subsequent deposits. Another author worry is about infringing agreed copyright agreements with publishers, yet only 10% of authors currently know of the SHERPA/RoMEO list of publisher permissions policies with respect to self-archiving, where clear guidance as to what a publisher permits is provided. Where it is not known if permission is required, however, authors are not seeking it and are self-archiving without it. Communicating their results to peers remains the primary reason for scholars publishing their work; in other words, researchers publish to have an impact on their field. The vast majority of authors (81%) responded that they would willingly comply with a mandate from their employer or research funder to deposit copies of their articles in an institutional or subject-based repository. A further 13% would comply reluctantly; 5% would not comply with such a mandate. In a separate exercise we asked the American Physical Society (APS) and the Institute of Physics Publishing Ltd (IOPP) what their experiences have been over the 14 years that arXiv has been in existence. How many subscriptions have been lost as a result of arXiv? Both societies said they could not identify any losses of subscriptions for this reason and that they do not view arXiv as a threat to their business (rather the opposite -- in fact the APS helped establish an arXiv mirror site at the Brookhaven National Laboratory)."
2071195592,The weakening relationship between the impact factor and papers' citations in the digital age,"Historically, papers have been physically bound to the journal in which they were published; but in the digital age papers are available individually, no longer tied to their respective journals. Hence, papers now can be read and cited based on their own merits, independently of the journal's physical availability, reputation, or impact factor ( IF ). We compare the strength of the relationship between journals' IFs and the actual citations received by their respective papers from 1902 to 2009. Throughout most of the 20th century, papers' citation rates were increasingly linked to their respective journals' IFs . However, since 1990, the advent of the digital age, the relation between IFs and paper citations has been weakening. This began first in physics, a field that was quick to make the transition into the electronic domain. Furthermore, since 1990 the overall proportion of highly cited papers coming from highly cited journals has been decreasing and, of these highly cited papers, the proportion not coming from highly cited journals has been increasing. Should this pattern continue, it might bring an end to the use of the IF as a way to evaluate the quality of journals, papers, and researchers."
2154053722,Tradition and Innovation in Scientists’ Research Strategies,"What factors affect a scientist’s choice of research problem? Qualitative research in the history and sociology of science suggests that this choice is patterned by an “essential tension” between productive tradition and risky innovation. We examine this tension through Bourdieu’s field theory of science, and we explore it empirically by analyzing millions of biomedical abstracts from MEDLINE. We represent the evolving state of chemical knowledge with networks extracted from these abstracts. We then develop a typology of research strategies on these networks. Scientists can introduce novel chemicals and chemical relationships (innovation) or delve deeper into known ones (tradition). They can consolidate knowledge clusters or bridge them. The aggregate distribution of published strategies remains remarkably stable. High-risk innovation strategies are rare and reflect a growing focus on established knowledge. An innovative publication is more likely to achieve high impact than a conservative one, but the additional reward does not compensate for the risk of failing to publish. By studying prizewinners in biomedicine and chemistry, we show that occasional gambles for extraordinary impact are a compelling explanation for observed levels of risky innovation. Our analysis of the essential tension identifies institutional forces that sustain tradition and suggests policy interventions to foster innovation."
1517593287,The rise and rise of citation analysis,"It is a sobering fact that some 90% of papers that have been published in academic journals are never cited. Indeed, as many as 50% of papers are never read by anyone other than their authors, referees and journal editors. We know this thanks to citation analysis, a branch of information science in which researchers study the way articles in a scholarly field are accessed and referenced by others."
2169420254,Who reads research articles? An altmetrics analysis of Mendeley user categories,"Little detailed information is known about who reads research articles and the contexts in which research articles are read. Using data about people who register in M endeley as readers of articles, this article explores different types of users of C linical M edicine, E ngineering and T echnology, S ocial S cience, P hysics, and C hemistry articles inside and outside academia. The majority of readers for all disciplines were PhD students, postgraduates, and postdocs but other types of academics were also represented. In addition, many C linical M edicine articles were read by medical professionals. The highest correlations between citations and M endeley readership counts were found for types of users who often authored academic articles, except for associate professors in some sub‐disciplines. This suggests that M endeley readership can reflect usage similar to traditional citation impact if the data are restricted to readers who are also authors without the delay of impact measured by citation counts. At the same time, M endeley statistics can also reveal the hidden impact of some research articles, such as educational value for nonauthor users inside academia or the impact of research articles on practice for readers outside academia."
1758588624,"ResearchGate: Disseminating, communicating, and measuring Scholarship?","R esearch G ate is a social network site for academics to create their own profiles, list their publications, and interact with each other. Like Academia.edu, it provides a new way for scholars to disseminate their work and hence potentially changes the dynamics of informal scholarly communication. This article assesses whether R esearch G ate usage and publication data broadly reflect existing academic hierarchies and whether individual countries are set to benefit or lose out from the site. The results show that rankings based on R esearch G ate statistics correlate moderately well with other rankings of academic institutions, suggesting that R esearch G ate use broadly reflects the traditional distribution of academic capital. Moreover, while B razil, I ndia, and some other countries seem to be disproportionately taking advantage of R esearch G ate, academics in C hina, S outh K orea, and R ussia may be missing opportunities to use R esearch G ate to maximize the academic impact of their publications."
1987311106,Scholarly Influence in the Field of Management: A Bibliometric Analysis of the Determinants of University and Author Impact in the Management Literature in the Past Quarter Century,"The purpose of this study is to identify the universities and research scholars who have had the greatest impact on the field of management during the past quarter century and the factors that influence their impact. Using bibliometric techniques, the authors examined 30 management journals to identify the 100 most-cited universities and 150 most-cited authors from 1981 to 2004. The analysis included more than 1,600 universities and 25,000 management scholars across five individual time periods. The findings showed that (a) a relatively small proportion of universities and scholars accounted for the majority of the citations in the field; (b) total publications accounted for the majority of the variance in university citations; (c) university size, the number of PhDs awarded, research expenditures, and endowment assets had the biggest impact on university publications; and (d) total publications, years in the field, graduate school reputation, and editorial board memberships had the biggest effect on a scholar's citations."
2088433402,Using Publication Metrics to Highlight Academic Productivity and Research Impact,"Abstract This article provides a broad overview of widely available measures of academic productivity and impact using publication data and highlights uses of these metrics for various purposes. Metrics based on publication data include measures such as number of publications, number of citations, the journal impact factor score, and the h‐index, as well as emerging metrics based on document‐level metrics. Publication metrics can be used for a variety of purposes for tenure and promotion, grant applications and renewal reports, benchmarking, recruiting efforts, and administrative purposes for departmental or university performance reports. The authors also highlight practical applications of measuring and reporting academic productivity and impact to emphasize and promote individual investigators, grant applications, or department output."
1859177121,Differences in impact factor across fields and over time,"Abstract The bibliometric measure impact factor is a leading indicator of journal influence, and impact factors are routinely used in making decisions ranging from selecting journal subscriptions to allocating research funding to deciding tenure cases. Yet journal impact factors have increased gradually over time, and moreover impact factors vary widely across academic disciplines. Here we quantify inflation over time and differences across fields in impact factor scores and determine the sources of these differences. We find that the average number of citations in reference lists has increased gradually, and this is the predominant factor responsible for the inflation of impact factor scores over time. Field‐specific variation in the fraction of citations to literature indexed by Thomson Scientific's Journal Citation Reports is the single greatest contributor to differences among the impact factors of journals in different fields. The growth rate of the scientific literature as a whole, and cross‐field differences in net size and growth rate of individual fields, have had very little influence on impact factor inflation or on cross‐field differences in impact factor."
3037055661,Citation Statistics,"This is a report about the use and misuse of citation data in the assessment of scientific research.The idea that research assessment must be done using ""simple and objective"" methods is increasingly prevalent today.The ""simple and objective"" methods are broadly interpreted as bibliometrics, that is, citation data and the statistics derived from them.There is a belief that citation statistics are inherently more accurate because they substitute simple numbers for complex judgments, and hence overcome the possible subjectivity of peer review.But this belief is unfounded.• Relying on statistics is not more accurate when the statistics are improperly used.Indeed, statistics can mislead when they are misapplied or misunderstood.Much of modern bibliometrics seems to rely on experience and intuition about the interpretation and validity of citation statistics.• While numbers appear to be ""objective,"" their objectivity can be illusory.The meaning of a citation can be even more subjective than peer review.Because this subjectivity is less obvious for citations, those who use citation data are less likely to understand their limitations."
2095455968,Effects of Editorial Peer Review,Context Editorial peer review is widely used to select submissions to journals for publication and is presumed to improve their usefulness.Sufficient research on peer re-
2172025811,Integrated impact indicators compared with impact factors: An alternative research design with policy implications,"Abstract In bibliometrics, the association of “impact” with central‐tendency statistics is mistaken. Impacts add up, and citation curves therefore should be integrated instead of averaged. For example, the journals MIS Quarterly and Journal of the American Society for Information Science and Technology differ by a factor of 2 in terms of their respective impact factors (IF), but the journal with the lower IF has the higher impact. Using percentile ranks (e.g., top‐1%, top‐10%, etc.), an Integrated Impact Indicator ( I3 ) can be based on integration of the citation curves, but after normalization of the citation curves to the same scale. The results across document sets can be compared as percentages of the total impact of a reference set. Total number of citations, however, should not be used instead because the shape of the citation curves is then not appreciated. I3 can be applied to any document set and any citation window. The results of the integration (summation) are fully decomposable in terms of journals or institutional units such as nations, universities, and so on because percentile ranks are determined at the paper level. In this study, we first compare I3 with IFs for the journals in two Institute for Scientific Information subject categories (“Information Science & Library Science” and “Multidisciplinary Sciences”). The library and information science set is additionally decomposed in terms of nations. Policy implications of this possible paradigm shift in citation impact analysis are specified."
3125019846,The Quest for Citations: Drivers of Article Impact,"Why do some articles become building blocks for future scholars, whereas others remain unnoticed? The authors aim to answer this question by contrasting, synthesizing, and simultaneously testing three scientometric perspectives—universalism, social constructivism, and presentation—on the influence of article and author characteristics on article citations. They study all articles published in a sample of five major journals in marketing from 1990 to 2002 that are central to the discipline. They count the number of citations each of these articles has received and regress this count on an extensive set of characteristics of the article (i.e., article quality, article domain, title length, the use of attention grabbers, and expositional clarity) and the author (i.e., author visibility and author personal promotion). They find that the number of citations an article in the marketing discipline receives depends more on “what is said” (quality and domain) and “who says it” (author visibility and personal promotion) than on “how it is said” (title length, the use of attention grabbers, and expositional clarity). The insights gleaned from this analysis contribute to the marketing literature and are relevant to scientific stakeholders, such as the management of scientific journals and individual academic scholars, as they strive to maximize citations. They are also relevant to marketing practitioners; they inform practitioners on characteristics of the academic journals in marketing and their relevance to decisions they face. Conversely, the insights also raise challenges regarding how to make journals accessible and relevant to marketing practitioners: (1) Authors visible to academics are not necessarily visible to practitioners; (2) the readability of an article may hurt academic credibility and impact, but it may be instrumental in influencing practitioners; and (3) it remains questionable whether articles that academics assess to be of high quality are also managerially relevant."
1994333330,The Structural Influence of Marketing Journals: A Citation Analysis of the Discipline and its Subareas over Time,"The authors investigate the overall and subarea influence of a comprehensive set of marketing and marketing-related journals at three points in time during a 30-year period using a citation-based measure of structural influence. The results show that a few journals wield a disproportionate amount of influence in the marketing journal network as a whole and that influential journals tend to derive their influence from many different journals. Different journals are most influential in different subareas of marketing; general business and managerially oriented journals have lost influence, whereas more specialized marketing journals have gained in influence over time. The Journal of Marketing emerges as the most influential marketing journal in the final period (1996–97) and as the journal with the broadest span of influence across all subareas. Yet the Journal of Marketing is notably influential among applied marketing journals, which themselves are of lesser influence. The index of structural influence is significantly correlated with other objective and subjective measures of influence but least so with the impact factors reported in the Social Sciences Citation Index. Overall, the findings demonstrate the rapid maturation of the marketing discipline and the changing role of key journals in the process."
2136074788,Free Labor for Costly Journals?,"There is a remarkable difference between the prices that commercial publishers charge to libraries for economics journals and the prices charged by professional societies and university presses. This price difference does not reflect a difference in quality. The six most-cited economics journals listed in the Social Science Citation Index are all nonprofit journals, and their library subscription prices average about $180 per year. Only five of the 20 most-cited journals are owned by commercial publishers, and the average price of these five journals is about $1660 per year. Tables 1 and 2 compare library costs and measures of cost-effectiveness for the ten most-cited nonprofit journals and the ten most-cited journals owned by commercial presses. The average price per page of the commercial journals is about six times as high and the average price per citation is about 16 times as high as for the nonprofit journals. In Tables 1 and 2, the first column shows the year 2001 library subscription price and the second column shows the price per page (calculated by dividing year 2001 price by the number of pages published in the year 2000). The third column reports the price per citation. This is the library subscription price divided by the number of times that articles in this journal were cited in 1998, as recorded by the Social Science Citation Index. The fourth column, price per recent citation, is the library subscription price divided by the number of times that the 1996 and 1997 volumes of the journal were cited in 1998. The citation rank is found by ranking journals according to the number of times that this journal was cited in 1998. The differences in prices and cost-effectiveness between nonprofit and commercial journals are similar for less prestigious journals. I have assembled a data"
2159336657,"How the Scientific Community Reacts to Newly Submitted Preprints: Article Downloads, Twitter Mentions, and Citations","We analyze the online response to the preprint publication of a cohort of 4,606 scientific articles submitted to the preprint database arXiv.org between October 2010 and May 2011. We study three forms of responses to these preprints: downloads on the arXiv.org site, mentions on the social media site Twitter, and early citations in the scholarly record. We perform two analyses. First, we analyze the delay and time span of article downloads and Twitter mentions following submission, to understand the temporal configuration of these reactions and whether one precedes or follows the other. Second, we run regression and correlation tests to investigate the relationship between Twitter mentions, arXiv downloads and article citations. We find that Twitter mentions and arXiv downloads of scholarly articles follow two distinct temporal patterns of activity, with Twitter mentions having shorter delays and narrower time spans than arXiv downloads. We also find that the volume of Twitter mentions is statistically correlated with arXiv downloads and early citations just months after the publication of a preprint, with a possible bias that favors highly mentioned articles."
4243702734,A Google Scholar h‐index for journals: An alternative metric to measure journal impact in economics and business,Abstract We propose a new data source (Google Scholar) and metric (Hirsch's h‐index) to assess journal impact in the field of economics and business. A systematic comparison between the Google Scholar h‐index and the ISI Journal Impact Factor for a sample of 838 journals in economics and business shows that the former provides a more accurate and comprehensive measure of journal impact.
1731700882,Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics,"Journal‐based citations are an important source of data for impact indices. However, the impact of journal articles extends beyond formal scholarly discourse. Measuring online scholarly impact calls for new indices, complementary to the older ones. This article examines a possible alternative metric source, blog posts aggregated at ResearchBlogging.org, which discuss peer‐reviewed articles and provide full bibliographic references. Articles reviewed in these blogs therefore receive “blog citations.” We hypothesized that articles receiving blog citations close to their publication time receive more journal citations later than the articles in the same journal published in the same year that did not receive such blog citations. Statistically significant evidence for articles published in 2009 and 2010 support this hypothesis for seven of 12 journals (58%) in 2009 and 13 of 19 journals (68%) in 2010. We suggest, based on these results, that blog citations can be used as an alternative metric source."
3001554335,"Scopus as a curated, high-quality bibliometric data source for academic research in quantitative science studies","Scopus is among the largest curated abstract and citation databases, with a wide global and regional coverage of scientific journals, conference proceedings, and books, while ensuring only the highest quality data are indexed through rigorous content selection and re-evaluation by an independent Content Selection and Advisory Board. Additionally, extensive quality assurance processes continuously monitor and improve all data elements in Scopus. Besides enriched metadata records of scientific articles, Scopus offers comprehensive author and institution profiles, obtained from advanced profiling algorithms and manual curation, ensuring high precision and recall. The trustworthiness of Scopus has led to its use as bibliometric data source for large-scale analyses in research assessments, research landscape studies, science policy evaluations, and university rankings. Scopus data have been offered for free for selected studies by the academic research community, such as through application programming interfaces, which have led to many publications employing Scopus data to investigate topics such as researcher mobility, network visualizations, and spatial bibliometrics. In June 2019, the International Center for the Study of Research was launched, with an advisory board consisting of bibliometricians, aiming to work with the scientometric research community and offering a virtual laboratory where researchers will be able to utilize Scopus data."
2592084954,Citation-based clustering of publications using CitNetExplorer and VOSviewer,"Clustering scientific publications in an important problem in bibliometric research. We demonstrate how two software tools, CitNetExplorer and VOSviewer, can be used to cluster publications and to analyze the resulting clustering solutions. CitNetExplorer is used to cluster a large set of publications in the field of astronomy and astrophysics. The publications are clustered based on direct citation relations. CitNetExplorer and VOSviewer are used together to analyze the resulting clustering solutions. Both tools use visualizations to support the analysis of the clustering solutions, with CitNetExplorer focusing on the analysis at the level of individual publications and VOSviewer focusing on the analysis at an aggregate level. The demonstration provided in this paper shows how a clustering of publications can be created and analyzed using freely available software tools. Using the approach presented in this paper, bibliometricians are able to carry out sophisticated cluster analyses without the need to have a deep knowledge of clustering techniques and without requiring advanced computer skills."
2048250974,Algorithmic procedure for finding semantically related journals,"Abstract Using citations, papers and references as parameters a relatedness factor (RF) is computed for a series of journals. Sorting these journals by the RF produces a list of journals most closely related to a specified starting journal. The method appears to select a set of journals that are semantically most similar to the target journal. The algorithmic procedure is illustrated for the journal Genetics . Inter‐journal citation data needed to calculate the RF were obtained from the 1996 ISI Journal Citation Reports on CD‐ROM© . Out of the thousands of candidate journals in JCR©, thirty have been selected. Some of them are different from the journals in the JCR category for genetics and heredity. The new procedure is unique in that it takes varying journal sizes into account."
2051213584,"Improving the peer-review process for grant applications: Reliability, validity, bias, and generalizability.","Peer review is a gatekeeper, the final arbiter of what is valued in academia, but it has been criticized in relation to traditional psychological research criteria of reliability, validity, generalizability, and potential biases. Despite a considerable literature, there is surprisingly little sound peer-review research examining these criteria or strategies for improving the process. This article summarizes the authors' research program with the Australian Research Council, which receives thousands of grant proposals from the social science, humanities, and science disciplines and reviews by assessors from all over the world. Using multilevel cross-classified models, the authors critically evaluated peer reviews of grant applications and potential biases associated with applicants, assessors, and their interaction (e.g., age, gender, university, academic rank, research team composition, nationality, experience). Peer reviews lacked reliability, but the only major systematic bias found involved the inflated, unreliable, and invalid ratings of assessors nominated by the applicants themselves. The authors propose a new approach, the reader system, which they evaluated with psychology and education grant proposals and found to be substantially more reliable and strategically advantageous than traditional peer reviews of grant applications."
2090174702,Publication fees in open access publishing: Sources of funding and factors influencing choice of journal,"Abstract Open access (OA) journals distribute their content at no charge and use other means of funding the publication process. Publication fees or article‐processing charges (APC)s have become the predominant means for funding professional OA publishing. We surveyed 1,038 authors who recently published articles in 74 OA journals that charge APCs stratified into seven discipline categories. Authors were asked about the source of funding for the APC, factors influencing their choice of a journal and past history publishing in OA and subscription journals. Additional information about the journal and the authors' country were obtained from the journal website. A total of 429 (41%) authors from 69 journals completed the survey. There were large differences in the source of funding among disciplines. Journals with impact factors charged higher APCs as did journals from disciplines where grant funding is plentiful. Fit, quality, and speed of publication were the most important factors in the authors' choice of a journal. OA was less important but a significant factor for many authors in their choice of a journal to publish. These findings are consistent with other research on OA publishing and suggest that OA publishing funded through APCs is likely to continue to grow."
3125814070,The Measurement of Intellectual Influence,"We examine the problem of measuring influence based on the information contained in the data on the communications between scholarly publications, judicial decisions, patents, web pages, and other entities.The measurement of influence is useful to address several empirical questions such as reputation, prestige, aspects of the diffusion of knowledge, the markets for scientists and scientific publications, the dynamics of innovation, ranking algorithms of search engines in the World Wide Web, and others.In this paper we ask why any given methodology is reasonable and informative applying the axiomatic method.We find that a unique ranking method can be characterized by means of five axioms: anonymity, invariance to citation intensity, weak homogeneity, weak consistency, and invariance to splitting of journals.This method is easily implementable and turns out to be different from those regularly used in social and natural sciences, arts and humanities, and computer science."
3124253902,The Burden of Knowledge and the “Death of the Renaissance Man”: Is Innovation Getting Harder?,"Journal Article The Burden of Knowledge and the “Death of the Renaissance Man”: Is Innovation Getting Harder? Get access Benjamin F. Jones Benjamin F. Jones Northwestern University and NBER Search for other works by this author on: Oxford Academic Google Scholar The Review of Economic Studies, Volume 76, Issue 1, January 2009, Pages 283–317, https://doi.org/10.1111/j.1467-937X.2008.00531.x Published: 01 January 2009 Article history Received: 01 May 2005 Accepted: 01 June 2008 Published: 01 January 2009"
2263556835,Bibliometrics: tracking research impact by selecting the appropriate metrics,"Traditionally, the success of a researcher is assessed by the number of publications he or she publishes in peer-reviewed, indexed, high impact journals. This essential yardstick, often referred to as the impact of a specific researcher, is assessed through the use of various metrics. While researchers may be acquainted with such matrices, many do not know how to use them to enhance their careers. In addition to these metrics, a number of other factors should be taken into consideration to objectively evaluate a scientist's profile as a researcher and academician. Moreover, each metric has its own limitations that need to be considered when selecting an appropriate metric for evaluation. This paper provides a broad overview of the wide array of metrics currently in use in academia and research. Popular metrics are discussed and defined, including traditional metrics and article-level metrics, some of which are applied to researchers for a greater understanding of a particular concept, including varicocele that is the thematic area of this Special Issue of Asian Journal of Andrology. We recommend the combined use of quantitative and qualitative evaluation using judiciously selected metrics for a more objective assessment of scholarly output and research impact."
2152513188,Reputation and impact in academic careers,"Significance Over a scientist’s career, a reputation is developed, a standing within a research community, based largely upon the quantity and quality of his/her publications. Here, we develop a framework for quantifying the influence author reputation has on a publication’s future impact. We find author reputation plays a key role in driving a paper’s citation count early in its citation life cycle, before a tipping point, after which reputation has much less influence relative to the paper’s citation count. In science, perceived quality, and decisions made based on those perceptions, is increasingly linked to citation counts. Shedding light on the complex mechanisms driving these quantitative measures facilitates not only better evaluation of scientific outputs but also a more transparent evaluation of the scientists producing them."
2170027555,Factors affecting citation rates of research articles,"This study examines whether there are some general trends across subject fields regarding the factors affecting the number of citations of articles, focusing especially on those factors that are not directly related to the quality or content of articles (extrinsic factors). For this purpose, from 6 selected subject fields (condensed matter physics, inorganic and nuclear chemistry, electric and electronic engineering, biochemistry and molecular biology, physiology, and gastroenterology), original articles published in the same year were sampled ( n = 230–240 for each field). Then, the citation counts received by the articles in relatively long citation windows (6 and 11 years after publication) were predicted by negative binomial multiple regression ( NBMR ) analysis for each field. Various article features about author collaboration, cited references, visibility, authors' achievements (measured by past publications and citedness), and publishing journals were considered as the explanatory variables of NBMR . Some generality across the fields was found with regard to the selected predicting factors and the degree of significance of these predictors. The P rice index was the strongest predictor of citations, and number of references was the next. The effects of number of authors and authors' achievement measures were rather weak."
2152361179,Citation Analysis May Severely Underestimate the Impact of Clinical Research as Compared to Basic Research,"Citation analysis has become an important tool for research performance assessment in the medical sciences. However, different areas of medical research may have considerably different citation practices, even within the same medical field. Because of this, it is unclear to what extent citation-based bibliometric indicators allow for valid comparisons between research units active in different areas of medical research.A visualization methodology is introduced that reveals differences in citation practices between medical research areas. The methodology extracts terms from the titles and abstracts of a large collection of publications and uses these terms to visualize the structure of a medical field and to indicate how research areas within this field differ from each other in their average citation impact.Visualizations are provided for 32 medical fields, defined based on journal subject categories in the Web of Science database. The analysis focuses on three fields: Cardiac & cardiovascular systems, Clinical neurology, and Surgery. In each of these fields, there turn out to be large differences in citation practices between research areas. Low-impact research areas tend to focus on clinical intervention research, while high-impact research areas are often more oriented on basic and diagnostic research.Popular bibliometric indicators, such as the h-index and the impact factor, do not correct for differences in citation practices between medical fields. These indicators therefore cannot be used to make accurate between-field comparisons. More sophisticated bibliometric indicators do correct for field differences but still fail to take into account within-field heterogeneity in citation practices. As a consequence, the citation impact of clinical intervention research may be substantially underestimated in comparison with basic and diagnostic research."
2396414759,The Altmetrics Collection,"What paper should I read next? Who should I talk to at a conference? Which research group should get this grant? Researchers and funders alike must make daily judgments on how to best spend their limited time and money–judgments that are becoming increasingly difficult as the volume of scholarly communication increases. Not only does the number of scholarly papers continue to grow, it is joined by new forms of communication from data publications to microblog posts.To deal with incoming information, scholars have always relied upon filters. At first these filters were manually compiled compendia and corpora of the literature. But by the mid-20th century, filters built on manual indexing began to break under the weight of booming postwar science production. Garfield [1] and others pioneered a solution: automated filters that leveraged scientists own impact judgments, aggregating citations as “pellets of peer recognition.” [2].These citation-based filters have dramatically grown in importance and have become the tenet of how research impact is measured. But, like manual indexing 60 years ago, they may today be failing to keep up with the literature’s growing volume, velocity, and diversity [3].Citations are heavily gamed [4]–[6] and are painfully slow to accumulate [7], and overlook increasingly important societal and clinical impacts [8]. Most importantly, they overlook new scholarly forms like datasets, software, and research blogs that fall outside of the scope of citable research objects. In sum, citations only reflect formal acknowledgment and thus they provide only a partial picture of the science system [9]. Scholars may discuss, annotate, recommend, refute, comment, read, and teach a new finding before it ever appears in the formal citation registry. We need new mechanisms to create a subtler, higher-resolution picture of the science system.The Quest for Better FiltersThe scientometrics community has not been blind to the limitations of citation measures, and has collectively proposed methods to gather evidence of broader impacts and provide more detail about the science system: tracking acknowledgements [10], patents [11], mentorships [12], news articles [8], usage in syllabuses [13], and many others, separately and in various combinations [14]. The emergence of the Web, a “nutrient-rich space for scholars” [15], has held particular promise for new filters and lenses on scholarly output. Webometrics researchers have uncovered evidence of informal impact by examining networks of hyperlinks and mentions on the broader Web [16]–[18]. An important strand of webometrics has also examined the properties of article download data [7], [19], [20].The last several years, however, have presented a promising new approach to gathering fine-grained impact data: tracking large-scale activity around scholarly products in online tools and environments. These tools and environments include, among others:social media like Twitter and Facebookonline reference managers like CiteULike, Zotero, and Mendeleycollaborative encyclopedias like Wikipediablogs, both scholarly and general-audiencescholarly social networks, like ResearchGate or Academia.educonference organization sites like Lanyrd.comGrowing numbers of scholars are using these and similar tools to mediate their interaction with the literature. In doing so, they are leaving valuable tracks behind them–tracks with potential to show informal paths of influence with unprecedented speed and resolution. Many of these tools offer open APIs, supporting large-scale, automated mining of online activities and conversations around research objects [21].Altmetrics [22], [23] is the study and use of scholarly impact measures based on activity in online tools and environments. The term has also been used to describe the metrics themselves–one could propose in plural a “set of new altmetrics.” Altmetrics is in most cases a subset of both scientometrics and webometrics; it is a subset of the latter in"
2950850257,Caveats for the use of citation indicators in research and journal evaluations,"Abstract Aging of publications, percentage of self‐citations, and impact vary from journal to journal within fields of science. The assumption that citation and publication practices are homogenous within specialties and fields of science is invalid. Furthermore, the delineation of fields and among specialties is fuzzy. Institutional units of analysis and persons may move between fields or span different specialties. The match between the citation index and institutional profiles varies among institutional units and nations. The respective matches may heavily affect the representation of the units. Non‐Institute of Scientific Information (ISI) journals are increasingly cornered into “transdisciplinary” Mode‐2 functions with the exception of specialist journals publishing in languages other than English. An “externally cited impact factor” can be calculated for these journals. The citation impact of non‐ISI journals will be demonstrated using Science and Public Policy as the example."
2099424129,A bibliometric chronicling of library and information science's first hundred years,"This paper presents a condensed history of Library and Information Science ( LIS ) over the course of more than a century using a variety of bibliometric measures. It examines in detail the variable rate of knowledge production in the field, shifts in subject coverage, the dominance of particular publication genres at different times, prevailing modes of production, interactions with other disciplines, and, more generally, observes how the field has evolved. It shows that, despite a striking growth in the number of journals, papers, and contributing authors, a decrease was observed in the field's market‐share of all social science and humanities research. Collaborative authorship is now the norm, a pattern seen across the social sciences. The idea of boundary crossing was also examined: in 2010, nearly 60% of authors who published in LIS also published in another discipline. This high degree of permeability in LIS was also demonstrated through reference and citation practices: LIS scholars now cite and receive citations from other fields more than from LIS itself. Two major structural shifts are revealed in the data: in 1960, LIS changed from a professional field focused on librarianship to an academic field focused on information and use; and in 1990, LIS began to receive a growing number of citations from outside the field, notably from Computer Science and Management, and saw a dramatic increase in the number of authors contributing to the literature of the field."
2463568293,How open science helps researchers succeed,"Open access, open data, open source and other open scholarship practices are growing in popularity and necessity. However, widespread adoption of these practices has not yet been achieved. One reason is that researchers are uncertain about how sharing their work will affect their careers. We review literature demonstrating that open research is associated with increases in citations, media attention, potential collaborators, job opportunities and funding opportunities. These findings are evidence that open research practices bring significant benefits to researchers relative to more traditional closed practices."
2785537869,Fifty years of the European Journal of Marketing: a bibliometric analysis,"Purpose The European Journal of Marketing was created in 1967. In 2017, the journal celebrates its 50th anniversary. Therefore, the purpose of this study is to present a bibliometric overview of the leading trends of the journal during this period. Design/methodology/approach This work uses the Scopus database to analyse the most productive authors, institutions and countries, as well as the most cited papers and the citing articles. The investigation uses bibliometric indicators to represent the bibliographic data, including the total number of publications and citations between 1967 and 2017. Additionally, the article also develops a graphical visualization of the bibliographic material by using the visualization of similarities viewer software to map journals, keywords and institutions with bibliographic coupling and co-citation analysis. Findings British authors and institutions are the most productive in the journal, although Australians’ are growing significantly the number of papers published. Continental European institutions are also increasing the number of publications, but they are still far from reaching the British contribution so far. In the mid-term, however, these zone’s authors and institutions, especially those from big European countries like France, Germany, Italy and Spain, should reach a closer performance to British ones; more as less long, historic, but more recent periods of analysis are considered. Practical implications This article is useful for any reader of this journal to understand questions such as papers’ European Journal of Marketing -related scientific productivity in terms of, for instance, contributors/authors, institutions and countries, or the main sources used to back them. Originality/value This is the first comprehensive article offering a general overview of the leading trends and researchers of the journal over its history."
2033069678,Article-Level Metrics and the Evolution of Scientific Impact,"Formally published papers that have been through a traditional prepublication peer review process remain the most important means of communicating science today. Researchers depend on them to learn about the latest advances in their fields and to report their own findings. The intentions of traditional peer review are certainly noble: to ensure methodological integrity and to comment on potential significance of experimental studies through examination by a panel of objective, expert colleagues. In principle, this system enables science to move forward on the collective confidence of previously published work. Unfortunately, the traditional system has inspired methods of measuring impact that are suboptimal for their intended uses."
2625580771,What I learned from predatory publishers,"This article is a first-hand account of the author's work identifying and listing predatory publishers from 2012 to 2017. Predatory publishers use the gold (author pays) open access model and aim to generate as much revenue as possible, often foregoing a proper peer review. The paper details how predatory publishers came to exist and shows how they were largely enabled and condoned by the open-access social movement, the scholarly publishing industry, and academic librarians. The author describes tactics predatory publishers used to attempt to be removed from his lists, details the damage predatory journals cause to science, and comments on the future of scholarly publishing."
4382986001,Article processing charges for open access journal publishing: A review,"Abstract Some open access (OA) publishers charge authors fees to make their articles freely available online. This paper reviews literature on article processing charges (APCs) that has been published since 2000. Despite praise for diamond OA journals, which charge no fees, most OA articles are published by commercial publishers that charge APCs. Publishers fix APCs depending on the reputation assigned to journals by peers. Evidence shows a relationship between high impact metrics and higher, faster rising APCs. Authors express reluctance about APCs, although this varies by discipline depending on previous experience of paying publication fees and the availability of research grants to cover them. Authors rely on a mix of research grants, library funds and personal assets to pay the charges. Two major concerns have been raised in relation to APCs: the inability of poorly funded authors to publish research and their impact on journal quality. Waivers have not solved the first issue. Research shows little extension of waiver use, unintended side effects on co‐author networks and concerns regarding criteria to qualify for them. Bibliometric studies concur that journals that charge APCs have a similar citation impact to journals that rely on other income sources."
1541009479,How professors think: inside the curious world of academic judgment,* Contents * Opening the Black Box of Peer Review * How Panels Work * On Disciplinary Cultures * Pragmatic Fairness: Customary Rules of Deliberation * Recognizing Various Kinds of Excellence * Considering Interdisciplinarity and Diversity * Implications in the United States and Abroad * Appendix: Methods and Data Analysis * Notes * References * Acknowledgments * Index
3154793279,Usage bibliometrics,"Scholarly usage data provides unique opportunities to address the known shortcomings of citation analysis. However, the collection, processing and analysis of usage data remains an area of active research. This article provides a review of the state-of-the-art in usage-based informetric, i.e. the use of usage data to study the scholarly process."
1482062756,"The ""impact factor"" revisited","Abstract The number of scientific journals has become so large that individuals, institutions and institutional libraries cannot completely store their physical content. In order to prioritize the choice of quality information sources, librarians and scientists are in need of reliable decision aids. The ""impact factor"" (IF) is the most commonly used assessment aid for deciding which journals should receive a scholarly submission or attention from research readership. It is also an often misunderstood tool. This narrative review explains how the IF is calculated, how bias is introduced into the calculation, which questions the IF can or cannot answer, and how different professional groups can benefit from IF use."
2011453004,Methods for accrediting publications to authors or countries: Consequences for evaluation studies,"One aim of science evaluation studies is to determine quantitatively the contribution of different players (authors, departments, countries) to the whole system. This information is then used to study the evolution of the system, for instance to gauge the results of special national or international programs. Taking articles as our basic data, we want to determine the exact relative contribution of each coauthor or each country. These numbers are then brought together to obtain country scores, or department scores, etc. It turns out, as we will show in this article, that different scoring methods can yield totally different rankings. In addition to this, a relative increase according to one method can go hand in hand with a relative decrease according to another counting method. Indeed, we present examples in which country (or author) c has a smaller relative score in the total counting system than in the fractional counting one, yet this smaller score has a higher importance than the larger one (fractional counting). Similar anomalies were constructed for total versus proportional counts and for total versus straight counts. Consequently, a ranking between countries, universities, research groups or authors, based on one particular accrediting method does not contain an absolute truth about their relative importance. Different counting methods should be used and compared. Differences are illustrated with a real-life example. Finally, it is shown that some of these anomalies can be avoided by using geometric instead of arithmetic averages."
2145617564,"What Causes a Management Article to be Cited—Article, Author, or Journal?","This article examines the influence of citation impact on scholarship. Citation analysis allows authors to determine how often their work has been cited by others. It has become a performance metric for scholars of the influence and relevance of their work. However, the authors contends there is very little research to understand citation rates. They conclude that the pressure from journals and universities for prestige is the key element driving citation rates. Increased information technology is expected to continue this trend into the future."
3101883599,"Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: a multidisciplinary comparison of coverage via citations","New sources of citation data have recently become available, such as Microsoft Academic, Dimensions, and the OpenCitations Index of CrossRef open DOI-to-DOI citations (COCI). Although these have been compared to the Web of Science Core Collection (WoS), Scopus, or Google Scholar, there is no systematic evidence of their differences across subject categories. In response, this paper investigates 3,073,351 citations found by these six data sources to 2,515 English-language highly-cited documents published in 2006 from 252 subject categories, expanding and updating the largest previous study. Google Scholar found 88% of all citations, many of which were not found by the other sources, and nearly all citations found by the remaining sources (89-94%). A similar pattern held within most subject categories. Microsoft Academic is the second largest overall (60% of all citations), including 82% of Scopus citations and 86% of WoS citations. In most categories, Microsoft Academic found more citations than Scopus and WoS (182 and 223 subject categories, respectively), but had coverage gaps in some areas, such as Physics and some Humanities categories. After Scopus, Dimensions is fourth largest (54% of all citations), including 84% of Scopus citations and 88% of WoS citations. It found more citations than Scopus in 36 categories, more than WoS in 185, and displays some coverage gaps, especially in the Humanities. Following WoS, COCI is the smallest, with 28% of all citations. Google Scholar is still the most comprehensive source. In many subject categories Microsoft Academic and Dimensions are good alternatives to Scopus and WoS in terms of coverage."
1993451046,Google Scholar: the pros and the cons,"Purpose To identify the pros and the cons of Google Scholar. Design/methodology/approach Chronicles the recent history of the Google Scholar search engine from its inception in November 2004 and critiques it with regard to its merits and demerits. Findings Feels that there are massive content omissions presently but that, with future changes in its structure, Google Scholar will become an excellent free tool for scholarly information discovery and retrieval. Originality/value Presents a useful analysis for potential users of the Google Scholar site."
2125660293,Electronic Publication and the Narrowing of Science and Scholarship,"Online journals promise to serve more information to more dispersed audiences and are more efficiently searched and recalled. But because they are used differently than print—scientists and scholars tend to search electronically and follow hyperlinks rather than browse or peruse—electronically available journals may portend an ironic change for science. Using a database of 34 million articles, their citations (1945 to 2005), and online availability (1998 to 2005), I show that as more journal issues came online, the articles referenced tended to be more recent, fewer journals and articles were cited, and more of those citations were to fewer journals and articles. The forced browsing of print archives may have stretched scientists and scholars to anchor findings deeply into past and present scholarship. Searching online is more efficient and following hyperlinks quickly puts researchers in touch with prevailing opinion, but this may accelerate consensus and narrow the range of findings and ideas built upon."
2076113539,"Gender, Family Characteristics, and Publication Productivity among Scientists","This paper concentrates upon the relationship between marriage, parental status, and publication productivity for women in academic science, with comparisons to men. Findings indicate that gender, family characteristics, and productivity are complex considerations that go beyond being married or not married, and the presence or absence of children. For women particularly, the relationship between marriage and productivity varies by type of marriage: first compared with subsequent marriage, and occupation of spouse (in scientific compared with non-scientific occupation). Further, type of family composition is important: women with preschool children have higher productivity than women without children or with school-age children. Women with preschool children are found to be a socially selective group in their characteristics, particularly in their allocations of time."
2468587249,Contributorship and division of labor in knowledge production,"Scientific authorship has been increasingly complemented with contributorship statements. While such statements are said to ensure more equitable credit and responsibility attribution, they also provide an opportunity to examine the roles and functions that authors play in the construction of knowledge and the relationship between these roles and authorship order. Drawing on a comprehensive and multidisciplinary dataset of 87,002 documents in which contributorship statements are found, this article examines the forms that division of labor takes across disciplines, the relationships between various types of contributions, as well as the relationships between the contribution types and various indicators of authors’ seniority. It shows that scientific work is more highly divided in medical disciplines than in mathematics, physics, and disciplines of the social sciences, and that, with the exception of medicine, the writing of the paper is the task most often associated with authorship. The results suggest a clear distinction between contributions that could be labeled as ‘technical’ and those that could be considered ‘conceptual’: While conceptual tasks are typically associated with authors with higher seniority, technical tasks are more often performed by younger scholars. Finally, results provide evidence of a U-shaped relationship between extent of contribution and author order: In all disciplines, first and last authors typically contribute to more tasks than middle authors. The paper concludes with a discussion of the implications of the results for the reward system of science."
1690291114,Does Interdisciplinary Research Lead to Higher Citation Impact? The Different Effect of Proximal and Distal Interdisciplinarity,"This article analyses the effect of degree of interdisciplinarity on the citation impact of individual publications for four different scientific fields. We operationalise interdisciplinarity as disciplinary diversity in the references of a publication, and rather than treating interdisciplinarity as a monodimensional property, we investigate the separate effect of different aspects of diversity on citation impact: i.e. variety, balance and disparity. We use a Tobit regression model to examine the effect of these properties of interdisciplinarity on citation impact, controlling for a range of variables associated with the characteristics of publications. We find that variety has a positive effect on impact, whereas balance and disparity have a negative effect. Our results further qualify the separate effect of these three aspects of diversity by pointing out that all three dimensions of interdisciplinarity display a curvilinear (inverted U-shape) relationship with citation impact. These findings can be interpreted in two different ways. On the one hand, they are consistent with the view that, while combining multiple fields has a positive effect in knowledge creation, successful research is better achieved through research efforts that draw on a relatively proximal range of fields, as distal interdisciplinary research might be too risky and more likely to fail. On the other hand, these results may be interpreted as suggesting that scientific audiences are reluctant to cite heterodox papers that mix highly disparate bodies of knowledge—thus giving less credit to publications that are too groundbreaking or challenging."
2158413506,The Research Excellence Framework and the 'impact agenda': are we creating a Frankenstein monster?,"Journal Article The Research Excellence Framework and the ‘impact agenda’: are we creating a Frankenstein monster? Get access Ben R Martin Ben R Martin Search for other works by this author on: Oxford Academic Google Scholar Research Evaluation, Volume 20, Issue 3, September 2011, Pages 247–254, https://doi.org/10.3152/095820211X13118583635693 Published: 01 September 2011"
2036452579,Self-citation corrections for the Hirsch index,"I propose to sharpen the index h, suggested by Hirsch as a useful index to characterize the scientific output of a researcher, by excluding the self-citations. Performing a self-experiment and also discussing in detail two anonymous data sets, it is shown that self-citations can significantly reduce the h index in contrast to Hirsch's expectations. This result is confirmed by an analysis of 13 further data sets."
1038774319,Citation analysis in research evaluation,"Het Institute for Scientific Information (ISI) introduceerde in 1964 de zogenaamde impact factor (IF), een grootheid die aangeeft hoe vaak het gemiddelde artikel in een bepaald wetenschappelijk tijdschrift gedurende een bepaalde periode na verschijning (gewoonlijk maximaal twee jaar) wordt aangehaald in andere wetenschappelijke publicaties. Dergelijke impactfactoren worden de laatste jaren steeds vaker gebruikt als instrument om inzicht te verkrijgen in de kwaliteit van wetenschappelijke tijdschriften, onderzoeksprogramma's, publicaties, individuele onderzoekers en onderzoeksgroepen. De achterliggende redenering is eenvoudig. Publicaties die vaak worden aangehaald zijn blijkbaar interessant voor een grote groep onderzoekers, wat suggereert dat die publicatie veel impact heeft op het onderzoek dat in een bepaald veld wordt verricht. Tijdschriften waarin veel van dergelijke veelaangehaalde artikelen verschijnen zijn dientengevolge van groot belang voor het veld; datzelfde geldt voor onderzoekers en onderzoeksgroepen die veel publicaties met een hoge impact realiseren. Kortom, impact is gelijk aan invloed is gelijk aan kwaliteit en prestige. Omdat de IF ook nog eens gemakkelijk en aan de hand van objectieve en duidelijke criteria te berekenen is, is deze grootheid een ideaal instrument om de kwaliteit van tijdschriften, onderzoekers en hun onderzoek in kaart te brengen en te beoordelen."
3103443220,Which Type of Citation Analysis Generates the Most Accurate Taxonomy of Scientific and Technical Knowledge?,"In 1965, Price foresaw the day when a citation‐based taxonomy of science and technology would be delineated and correspondingly used for science policy. A taxonomy needs to be comprehensive and accurate if it is to be useful for policy making, especially now that policy makers are utilizing citation‐based indicators to evaluate people, institutions and laboratories. Determining the accuracy of a taxonomy, however, remains a challenge. Previous work on the accuracy of partition solutions is sparse, and the results of those studies, although useful, have not been definitive. In this study we compare the accuracies of topic‐level taxonomies based on the clustering of documents using direct citation, bibliographic coupling, and co‐citation. Using a set of new gold standards—articles with at least 100 references—we find that direct citation is better at concentrating references than either bibliographic coupling or co‐citation. Using the assumption that higher concentrations of references denote more accurate clusters, direct citation thus provides a more accurate representation of the taxonomy of scientific and technical knowledge than either bibliographic coupling or co‐citation. We also find that discipline‐level taxonomies based on journal schema are highly inaccurate compared to topic‐level taxonomies, and recommend against their use."
1987684442,Citation Classics in Anesthetic Journals,"In Brief The number of citations an article receives after its publication reflects its recognition in the scientific community. In the present study, therefore, we identified and examined the characteristics of the top 100 most frequently cited articles published in anesthetic journals. These articles were identified using the database of the Science Citation Index Expanded (SCI-EXPANDED, 1945 to present) and the Web of SCIENCE®. The most-cited article received 707 citations and the least cited article received 197 citations, with a mean of 283 citations per article. These citation classics were published between 1954 and 1997 in 5 high-impact anesthetic journals, led by Anesthesiology (73 articles) followed by Anesthesia & Analgesia (10), British Journal of Anesthesia (10), Anesthesia (6), and Acta Anaesthesiologica Scandinavica (2). Seventy-eight articles were original publications, 22 were review articles, and one was an editorial. They originated from nine countries, with the United States contributing 70 articles. Within the United States, California leads the list of citation classics with 25 articles. Twenty-nine persons authored two or more of the top-cited articles. The main topics covered by the top-cited articles are pharmacology, volatile anesthetics, circulation, regional anesthesia, and lung physiology. This analysis of citation rates allows for the recognition of seminal advances in anesthesia and gives a historic perspective on the scientific progress of this specialty. IMPLICATIONS: We performed a citation analysis to identify important contributions and contributors to the anesthetic literature. These classic articles have influenced many people and have brought to our attention the many important advances in anesthesia made during the last 50 yr."
2147988994,An emerging view of scientific collaboration: Scientists' perspectives on collaboration and factors that impact collaboration,"Abstract Collaboration is often a critical aspect of scientific research, which is dominated by complex problems, rapidly changing technology, dynamic growth of knowledge, and highly specialized areas of expertise. An individual scientist can seldom provide all of the expertise and resources necessary to address complex research problems. This paper describes collaboration among a group of scientists, and considers how their experiences are socially shaped. The scientists were members of a newly formed distributed, multi‐disciplinary academic research center that was organized into four multi‐disciplinary research groups. Each group had 14 to 34 members, including faculty, postdoctoral fellows and students, at four geographically dispersed universities. To investigate challenges that emerge in establishing scientific collaboration, data were collected about members' previous and current collaborative experiences, perceptions regarding collaboration, and work practices during the center's first year of operation. The data for the study includes interviews with members of the center, observations of videoconferences and meetings, and a center‐wide sociometric survey. Data analysis has led to the development of a framework that identifies forms of collaboration that emerged among scientists (e.g., complementary and integrative collaboration) and associated factors, which influenced collaboration including personal compatibility, work connections, incentives, and infrastructure. These results may inform the specification of social and organizational practices, which are needed to establish collaboration in distributed, multi‐disciplinary research centers."
2599784308,Science Mapping: A Systematic Review of the Literature,"Abstract Purpose We present a systematic review of the literature concerning major aspects of science mapping to serve two primary purposes: First, to demonstrate the use of a science mapping approach to perform the review so that researchers may apply the procedure to the review of a scientific domain of their own interest, and second, to identify major areas of research activities concerning science mapping, intellectual milestones in the development of key specialties, evolutionary stages of major specialties involved, and the dynamics of transitions from one specialty to another. Design/methodology/approach We first introduce a theoretical framework of the evolution of a scientific specialty. Then we demonstrate a generic search strategy that can be used to construct a representative dataset of bibliographic records of a domain of research. Next, progressively synthesized co-citation networks are constructed and visualized to aid visual analytic studies of the domain’s structural and dynamic patterns and trends. Finally, trajectories of citations made by particular types of authors and articles are presented to illustrate the predictive potential of the analytic approach. Findings The evolution of the science mapping research involves the development of a number of interrelated specialties. Four major specialties are discussed in detail in terms of four evolutionary stages: conceptualization, tool construction, application, and codification. Underlying connections between major specialties are also explored. The predictive analysis demonstrates citations trajectories of potentially transformative contributions. Research limitations The systematic review is primarily guided by citation patterns in the dataset retrieved from the literature. The scope of the data is limited by the source of the retrieval, i.e. the Web of Science, and the composite query used. An iterative query refinement is possible if one would like to improve the data quality, although the current approach serves our purpose adequately. More in-depth analyses of each specialty would be more revealing by incorporating additional methods such as citation context analysis and studies of other aspects of scholarly publications. Practical implications The underlying analytic process of science mapping serves many practical needs, notably bibliometric mapping, knowledge domain visualization, and visualization of scientific literature. In order to master such a complex process of science mapping, researchers often need to develop a diverse set of skills and knowledge that may span multiple disciplines. The approach demonstrated in this article provides a generic method for conducting a systematic review. Originality/value Incorporating the evolutionary stages of a specialty into the visual analytic study of a research domain is innovative. It provides a systematic methodology for researchers to achieve a good understanding of how scientific fields evolve, to recognize potentially insightful patterns from visually encoded signs, and to synthesize various information so as to capture the state of the art of the domain."
2050025702,"Assessing the citation impact of books: The role of Google Books, Google Scholar, and Scopus","Abstract Citation indictors are increasingly used in some subject areas to support peer review in the evaluation of researchers and departments. Nevertheless, traditional journal‐based citation indexes may be inadequate for the citation impact assessment of book‐based disciplines. This article examines whether online citations from Google Books and Google Scholar can provide alternative sources of citation evidence. To investigate this, we compared the citation counts to 1,000 books submitted to the 2008 U.K. Research Assessment Exercise (RAE) from Google Books and Google Scholar with Scopus citations across seven book‐based disciplines (archaeology; law; politics and international studies; philosophy; sociology; history; and communication, cultural, and media studies). Google Books and Google Scholar citations to books were 1.4 and 3.2 times more common than were Scopus citations, and their medians were more than twice and three times as high as were Scopus median citations, respectively. This large number of citations is evidence that in book‐oriented disciplines in the social sciences, arts, and humanities, online book citations may be sufficiently numerous to support peer review for research evaluation, at least in the United Kingdom."
2167792912,An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision‐Makers,Scientific‐funding bodies are increasingly under pressure to use journal rankings to measure research quality. Hiring and promotion committees routinely hear an equivalent argument: ‘this is important work because it is to be published in prestigious journal X’. But how persuasive is such an argument? This paper examines data on citations to articles published 25 years ago. It finds that it is better to write the best article published in an issue of a medium‐quality journal such as the OBES than all four of the worst four articles published in an issue of an elite journal like the AER . Decision‐makers need to understand this.
4234189662,Modifying the journal impact factor by fractional citation weighting: The audience factor,"Abstract A new approach to the field normalization of the classical journal impact factor is introduced. This approach, called the audience factor , takes into consideration the citing propensity of journals for a given cited journal, specifically, the mean number of references of each citing journal, and fractionally weights the citations from those citing journals. Hence, the audience factor is a variant of a fractional citation‐counting scheme, but computed on the citing journal rather than the citing article or disciplinary level, and, in contrast to other cited‐side normalization strategies, is focused on the behavior of the citing entities. A comparison with standard journal impact factors from Thomson Reuters shows a more diverse representation of fields within various quintiles of impact, significant movement in rankings for a number of individual journals, but nevertheless a high overall correlation with standard impact factors."
4245802241,A multi-disciplinary perspective on emergent and future innovations in peer review,"Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure? Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research. With the advent of Web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review. We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation. Ideally, any new systems will demonstrate that they out-perform current models while avoiding as many of the biases of existing systems as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages. We also propose a novel hybrid platform model that, at least partially, resolves many of the technical and social issues associated with peer review, and can potentially disrupt the entire scholarly communication system. Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments."
2117689677,Academia.edu: Social network or Academic Network?,"Academic social network sites A cademia.edu and R esearchGate, and reference sharing sites M endeley, B ibsonomy, Z otero, and C ite UL ike, give scholars the ability to publicize their research outputs and connect with each other. With millions of users, these are a significant addition to the scholarly communication and academic information‐seeking eco‐structure. There is thus a need to understand the role that they play and the changes, if any, that they can make to the dynamics of academic careers. This article investigates attributes of philosophy scholars on A cademia.edu, introducing a median‐based, time‐normalizing method to adjust for time delays in joining the site. In comparison to students, faculty tend to attract more profile views but female philosophers did not attract more profile views than did males, suggesting that academic capital drives philosophy uses of the site more than does friendship and networking. Secondary analyses of law, history, and computer science confirmed the faculty advantage (in terms of higher profile views) except for females in law and females in computer science. There was also a female advantage for both faculty and students in law and computer science as well as for history students. Hence, A cademia.edu overall seems to reflect a hybrid of scholarly norms (the faculty advantage) and a female advantage that is suggestive of general social networking norms. Finally, traditional bibliometric measures did not correlate with any Academia.edu metrics for philosophers, perhaps because more senior academics use the site less extensively or because of the range informal scholarly activities that cannot be measured by bibliometric methods."
2130719266,Making Open Access work,"Purpose – The purpose of this paper is to provide an overview of one of the most important and controversial areas of scholarly communication: Open Access publishing and dissemination of research outputs. It identifies and discusses recent trends and future challenges for various stakeholders in delivering Open Access (OA) to the scholarly literature. Design/methodology/approach – The study is based on a number of interrelated strands of evidence which make up the current discourse on OA, comprising the peer-reviewed literature, grey literature and other forms of communication (including blogs and e-mail discussion lists). It uses a large-scale textual analysis of the peer-reviewed literature since 2010 (carried out using the VOSviewer tool) as a basis for discussion of issues raised in the OA discourse. Findings – A number of key themes are identified, including the relationship between “Green” OA (deposit in repositories) and “Gold” OA (OA journal publication), the developing evidence base associated with OA, researcher attitudes and behaviours, policy directions, management of repositories, development of journals, institutional responses and issues around impact and scholarly communication futures. It suggests that current challenges now focus on how OA can be made to work in practice, having moved on from the discussion of whether it should happen at all. Originality/value – The paper provides a structured evidence-based review of major issues in the OA field, and suggests key areas for future research and policy development."
3174975695,Is the open access citation advantage real? A systematic review of the citation of open access and subscription-based articles,"Over the last two decades, the existence of an open access citation advantage (OACA)-increased citation of articles made available open access (OA)-has been the topic of much discussion. While there has been substantial research to address this question, findings have been contradictory and inconclusive. We conducted a systematic review to compare studies of citations to OA and non-OA articles.A systematic search of 17 databases attempted to capture all relevant studies authored since 2001. The protocol was registered in Open Science Framework. We included studies with a direct comparison between OA and non-OA items and reported article-level citation as an outcome. Both randomized and non-randomized studies were included. No limitations were placed on study design, language, or publication type.A total of 5,744 items were retrieved. Ultimately, 134 items were identified for inclusion. 64 studies (47.8%) confirmed the existence of OACA, while 37 (27.6%) found that it did not exist, 32 (23.9%) found OACA only in subsets of their sample, and 1 study (0.8%) was inconclusive. Studies with a focus on multiple disciplines were significantly positively associated with finding that OACA exists in subsets, and are less associated with finding that OACA did not exist. In the critical appraisal of the included studies, 3 were found to have an overall low risk of bias. Of these, one found that an OACA existed, one found that it did not, and one found that an OACA occurred in subsets.As seen through the large number of studies identified for this review, OACA is a topic of continuing interest. Quality and heterogeneity of the component studies pose challenges for generalization. The results suggest the need for reporting guidelines for bibliometrics studies."
1560783210,Anatomy of green open access,"Open access ( OA ) is free, unrestricted access to electronic versions of scholarly publications. For peer‐reviewed journal articles, there are two main routes to OA : publishing in OA journals (gold OA ) or archiving of article copies or manuscripts at other web locations (green OA ). This study focuses on summarizing and extending current knowledge about green OA . A synthesis of previous studies indicates that green OA coverage of all published journal articles is approximately 12%, with substantial disciplinary variation. Typically, green OA copies become available after considerable time delays, partly caused by publisher‐imposed embargo periods, and partly by author tendencies to archive manuscripts only periodically. Although green OA copies should ideally be archived in proper repositories, a large share is stored on home pages and similar locations, with no assurance of long‐term preservation. Often such locations contain exact copies of published articles, which may infringe on the publisher's exclusive rights. The technical foundation for green OA uploading is becoming increasingly solid largely due to the rapid increase in the number of institutional repositories. The number of articles within the scope of OA mandates, which strongly influence the self‐archival rate of articles, is nevertheless still low."
2802509651,The Matthew effect in science funding,"A classic thesis is that scientific achievement exhibits a ""Matthew effect"": Scientists who have previously been successful are more likely to succeed again, producing increasing distinction. We investigate to what extent the Matthew effect drives the allocation of research funds. To this end, we assembled a dataset containing all review scores and funding decisions of grant proposals submitted by recent PhDs in a €2 billion granting program. Analyses of review scores reveal that early funding success introduces a growing rift, with winners just above the funding threshold accumulating more than twice as much research funding (€180,000) during the following eight years as nonwinners just below it. We find no evidence that winners' improved funding chances in subsequent competitions are due to achievements enabled by the preceding grant, which suggests that early funding itself is an asset for acquiring later funding. Surprisingly, however, the emergent funding gap is partly created by applicants, who, after failing to win one grant, apply for another grant less often."
1534953242,Use of the h index in neurosurgery,"Assessing academic productivity through simple quantification may overlook key information, and the use of statistical enumeration of academic output is growing. The h index, which incorporates both the total number of publications and the citations of those publications, has been recently proposed as an objective measure of academic productivity. The authors used several tools to calculate the h index for academic neurosurgeons to provide a basis for evaluating publishing by physicians.The h index of randomly selected academic neurosurgeons from a sample of one-third of the academic programs in the US was calculated using data from Google Scholar and from the Scopus database. The mean h index for each academic rank was determined. The h indices were also correlated with various other factors (such as time spent practicing neurosurgery, authorship position) to identify how these factors influenced the h index. The h indices were then compared with other citation statistics to evaluate the robustness of this metric. Finally, h indices were also calculated for a sampling of physicians in other medical specialties for comparison.As expected, the h index increased with academic rank and there was a statistically significant difference between each rank. A weighting based on position of authorship did not affect h indices. The h index was positively correlated with time since American Board of Neurological Surgery certification, and it was also correlated with other citation metrics. A comparison among medical specialties supports the assertion that h index values may not be comparable between fields, even closely related specialties.The h index appears to be a robust statistic for comparing academic output of neurosurgeons. Within the field of academic neurosurgery, clear differences of h indices between academic ranks exist. On average, an increase of the h index by 5 appears to correspond to the next highest academic rank, with the exception of chairperson. The h index can be used as a tool, along with other evaluations, to evaluate an individual's productivity in the academic advancement process within the field of neurosurgery but should not be used for comparisons across medical specialties."
2093498562,San Francisco Declaration on Research Assessment,"Summary On 16 December 2012, a group of editors and publishers of scholarly journals gathered at the Annual Meeting of The American Society for Cell Biology in San Francisco, CA, USA to discuss current issues related to how the quality of research output is evaluated and how the primary scientific literature is cited. The outcome of the meeting and further discussions is a set of recommendations that is referred to as the San Francisco Declaration on Research Assessment, published in May 2013. Disease Models & Mechanisms (DMM) fully supports this initiative. In concordance with the recommendations of the Declaration, DMM provides impact factor alongside a variety of other journal-based metrics, requests an author contribution statement for all research articles, places no restrictions on the reuse of reference lists, and has no limitation on the number of references in research articles."
2088909840,Open Access and Global Participation in Science,"Previous investigations into the impact of open-access journals on subsequent citations confounded open and electronic access and failed to track availability over time. With new data, we separated these effects. We demonstrate that a journal receives a modest increase in citations when it comes online freely, but the jump is larger when it first comes online through commercial sources. This effect reverses for poor countries where free-access articles are much more likely to be cited. Together, findings suggest that free Internet access widens the circle of those who read and make use of scientists' investigations."
2609499779,What is open peer review? A systematic review,"Background: “Open peer review” (OPR), despite being a major pillar of Open Science, has neither a standardized definition nor an agreed schema of its features and implementations. The literature reflects this, with numerous overlapping and contradictory definitions. While for some the term refers to peer review where the identities of both author and reviewer are disclosed to each other, for others it signifies systems where reviewer reports are published alongside articles. For others it signifies both of these conditions, and for yet others it describes systems where not only “invited experts” are able to comment. For still others, it includes a variety of combinations of these and other novel methods.Methods: Recognising the absence of a consensus view on what open peer review is, this article undertakes a systematic review of definitions of “open peer review” or “open review”, to create a corpus of 122 definitions. These definitions are systematically analysed to build a coherent typology of the various innovations in peer review signified by the term, and hence provide the precise technical definition currently lacking.Results: This quantifiable data yields rich information on the range and extent of differing definitions over time and by broad subject area. Quantifying definitions in this way allows us to accurately portray exactly how ambiguously the phrase “open peer review” has been used thus far, for the literature offers 22 distinct configurations of seven traits, effectively meaning that there are 22 different definitions of OPR in the literature reviewed.Conclusions: I propose a pragmatic definition of open peer review as an umbrella term for a number of overlapping ways that peer review models can be adapted in line with the aims of Open Science, including making reviewer and author identities open, publishing review reports and enabling greater participation in the peer review process."
4382462089,Factors associating with or predicting more cited or higher quality journal articles: An Annual Review of Information Science and Technology (ARIST) paper,"Abstract Identifying factors that associate with more cited or higher quality research may be useful to improve science or to support research evaluation. This article reviews evidence for the existence of such factors in article text and metadata. It also reviews studies attempting to estimate article quality or predict long‐term citation counts using statistical regression or machine learning for journal articles or conference papers. Although the primary focus is on document‐level evidence, the related task of estimating the average quality scores of entire departments from bibliometric information is also considered. The review lists a huge range of factors that associate with higher quality or more cited research in some contexts (fields, years, journals) but the strength and direction of association often depends on the set of papers examined, with little systematic pattern and rarely any cause‐and‐effect evidence. The strongest patterns found include the near universal usefulness of journal citation rates, author numbers, reference properties, and international collaboration in predicting (or associating with) higher citation counts, and the greater usefulness of citation‐related information for predicting article quality in the medical, health and physical sciences than in engineering, social sciences, arts, and humanities."
3121315213,Measuring academic influence: Not all citations are equal,"The importance of a research article is routinely measured by counting how many times it has been cited. However, treating all citations with equal weight ignores the wide variety of functions that citations perform. We want to automatically identify the subset of references in a bibliography that have a central academic influence on the citing paper. For this purpose, we examine the effectiveness of a variety of features for determining the academic influence of a citation. By asking authors to identify the key references in their own work, we created a data set in which citations were labeled according to their academic influence. Using automatic feature selection with supervised machine learning, we found a model for predicting academic influence that achieves good performance on this data set using only four features. The best features, among those we evaluated, were those based on the number of times a reference is mentioned in the body of a citing paper. The performance of these features inspired us to design an influence-primed h-index (the hip-index). Unlike the conventional h-index, it weights citations by how many times a reference is mentioned. According to our experiments, the hip-index is a better indicator of researcher performance than the conventional h-index."
2761801637,"Citation-Based Journal Rankings: Key Questions, Metrics, and Data Sources","This guide presents nine key questions that can help researchers make good use of citation-based journal rankings (metrics) in the natural and social sciences. The nine questions address the characteristics that distinguish one metric from another: the source documents, the citation-counting window, the document types counted, the cited-document window, the impact of highly cited documents, the treatment of self-citations, the distinction between size-dependent and size-independent metrics, the use of normalization to account for disciplinary differences in impact, and the use of weighting to account for the impact or centrality of each citing journal. Next, the guide reviews 19 standard citation metrics, including the h index, g index, impact factor, source normalized impact per paper, eigenfactor, article influence score, and SCImago journal rank. Three underlying data sources (Web of Science, Scopus, and Google Scholar) are described, along with six major data download sites: Journal Citation Reports, Eigenfactor, CWTS Journal Indicators, SCImago, Scopus Journal Metrics, Cabell's International, and Google Scholar Metrics. The paper summarizes the main criticisms of citation metrics and concludes with suggestions for their further development, dissemination, and use."
2950903507,Applying centrality measures to impact analysis: A coauthorship network analysis,"Abstract Many studies on coauthorship networks focus on network topology and network statistical mechanics. This article takes a different approach by studying micro‐level network properties with the aim of applying centrality measures to impact analysis. Using coauthorship data from 16 journals in the field of library and information science (LIS) with a time span of 20 years (1988–2007), we construct an evolving coauthorship network and calculate four centrality measures (closeness centrality, betweenness centrality, degree centrality, and PageRank) for authors in this network. We find that the four centrality measures are significantly correlated with citation counts. We also discuss the usability of centrality measures in author ranking and suggest that centrality measures can be useful indicators for impact analysis."
2158550208,Incentives and creativity: evidence from the academic life sciences,"Despite its presumed role as an engine of economic growth, we know surprisingly little about the drivers of scientific creativity. We exploit key differences across funding streams within the academic life sciences to estimate the impact of incentives on the rate and direction of scientific exploration. Specifically, we study the careers of investigators of the Howard Hughes Medical Institute (HHMI), which tolerates early failure, rewards long-term success, and gives its appointees great freedom to experiment, and grantees from the National Institutes of Health (NIH), who are subject to short review cycles, predefined deliverables, and renewal policies unforgiving of failure. Using a combination of propensity-score weighting and difference-in-differences estimation strategies, we find that HHMI investigators produce high-impact articles at a much higher rate than a control group of similarly accomplished NIH-funded scientists. Moreover, the direction of their research changes in ways that suggest the program induces them to explore novel lines of inquiry."
2212838023,Reproducibility of peer review in clinical neuroscience: Is agreement between reviewers any greater than would be expected by chance alone?,"We aimed to determine the reproducibility of assessments made by independent reviewers of papers submitted for publication to clinical neuroscience journals and abstracts submitted for presentation at clinical neuroscience conferences. We studied two journals in which manuscripts were routinely assessed by two reviewers, and two conferences in which abstracts were routinely scored by multiple reviewers. Agreement between the reviewers as to whether manuscripts should be accepted, revised or rejected was not significantly greater than that expected by chance [kappa = 0.08, 95% confidence interval (CI) -0.04 to -0.20] for 179 consecutive papers submitted to Journal A, and was poor (kappa = 0.28, 0.12 to 0. 40) for 116 papers submitted to Journal B. However, editors were very much more likely to publish papers when both reviewers recommended acceptance than when they disagreed or recommended rejection (Journal A, odds ratio = 73, 95% CI = 27 to 200; Journal B, 51, 17 to 155). There was little or no agreement between the reviewers as to the priority (low, medium, or high) for publication (Journal A, kappa = -0.12, 95% CI -0.30 to -0.11; Journal B, kappa = 0.27, 0.01 to 0.53). Abstracts submitted for presentation at the conferences were given a score of 1 (poor) to 6 (excellent) by multiple independent reviewers. For each conference, analysis of variance of the scores given to abstracts revealed that differences between individual abstracts accounted for only 10-20% of the total variance of the scores. Thus, although recommendations made by reviewers have considerable influence on the fate of both papers submitted to journals and abstracts submitted to conferences, agreement between reviewers in clinical neuroscience was little greater than would be expected by chance alone."
2170328207,The Gender Citation Gap in International Relations,"Abstract This article investigates the extent to which citation and publication patterns differ between men and women in the international relations (IR) literature. Using data from the Teaching, Research, and International Policy project on peer-reviewed publications between 1980 and 2006, we show that women are systematically cited less than men after controlling for a large number of variables including year of publication, venue of publication, substantive focus, theoretical perspective, methodology, tenure status, and institutional affiliation. These results are robust to a variety of modeling choices. We then turn to network analysis to investigate the extent to which the gender of an article's author affects that article's relative centrality in the network of citations between papers in our sample. Articles authored by women are systematically less central than articles authored by men, all else equal. This is likely because (1) women tend to cite themselves less than men, and (2) men (who make up a disproportionate share of IR scholars) tend to cite men more than women. This is the first study in political science to reveal significant gender differences in citation patterns and is especially meaningful because citation counts are increasingly used as a key measure of research's quality and impact."
1547304362,The Google scholar experiment: How to index false papers and manipulate bibliometric indicators,"G oogle S cholar has been well received by the research community. Its promises of free, universal, and easy access to scientific literature coupled with the perception that it covers the social sciences and the humanities better than other traditional multidisciplinary databases have contributed to the quick expansion of G oogle S cholar C itations and G oogle S cholar M etrics: 2 new bibliometric products that offer citation data at the individual level and at journal level. In this article, we show the results of an experiment undertaken to analyze G oogle S cholar's capacity to detect citation‐counting manipulation. For this, we uploaded 6 documents to an institutional web domain that were authored by a fictitious researcher and referenced all the publications of the members of the EC 3 research group at the U niversity of G ranada. The detection by G oogle S cholar of these papers caused an outburst in the number of citations included in the G oogle S cholar C itations profiles of the authors. We discuss the effects of such an outburst and how it could affect the future development of such products, at both the individual level and the journal level, especially if G oogle S cholar persists with its lack of transparency."
1969232170,Authors and open access publishing,ABSTRACT Surveys were carried out to learn more about authors and open access publishing. Awareness of open access journals among those who had not published in them was quite high; awareness of ‘self‐archiving’ was less. For open access journal authors the most important reason for publishing in that way was the principle of free access; their main concerns were grants and impact. Authors who had not published in an open access journal attributed that to unfamiliarity with such journals. Forty per cent of authors have self‐archived their traditional journal articles and almost twice as many say they would do so if required to.
2112470296,"Trends in the Usage of ISI Bibliometric Data: Uses, Abuses, and Implications","Librarians rely on the Institute for Scientific Information's journal impact factor as a tool for selecting periodicals, primarily in scientific disciplines. A current trend is to use this data as a means for evaluating the performance of departments, institutions, and even researchers in academic institutions—a process that is now being tied to tenure and promotion—despite the fact that such usage can be misleading and prejudicial. This paper will highlight the history of the development of impact factors, describe the limitations in their use, and provide a critique of the usage of impact factors in academic settings."
2167023602,Open peer review: A randomised controlled trial,"Background Most scientific journals practise anonymous peer review. There is no evidence, however, that this is any better than an open system. Aims To evaluate the feasibility of an open peer review system. Method Reviewers for the British Journal of Psychiatry were asked whether they would agree to have their name revealed to the authors whose papers they review; 408 manuscripts assigned to reviewers who agreed were randomised to signed or unsigned groups. We measured review quality, tone, recommendation for publication and time taken to complete each review. Results A total of 245 reviewers (76%) agreed to sign. Signed reviews were of higher quality, were more courteous and took longer to complete than unsigned reviews. Reviewers who signed were more likely to recommend publication. Conclusions This study supports the feasibility of an open peer review system and identifies such a system's potential drawbacks."
2187587692,Web indicators for research evaluation. Part 1: Citations and links to academic articles from the Web,"El gran uso de la web por parte de muchos sectores de la sociedad ha creado el potencial para nuevos indicadores de impacto más amplios. Este artículo revisa la investigación sobre Google Scholar y Google Patents, servicios que pueden ser utilizados como fuente de indicadores de impacto de artículos académicos. También se examinan brevemente los métodos para extraer tipos de enlaces y citas de la Web en su conjunto, aunque para que sean útiles en la práctica los indicadores que éstos generan ahora son probablemente demasiado amplios y demasiado dominados por sitios web generados automáticamente, como catálogos de biblioteca y de editoriales. Indicadores basados en la web más valiosos se pueden derivar de determinados tipos de páginas web que citan investigaciones académicas, tales como presentaciones online, programas de cursos y blogs científicos. Éstos proporcionan evidencia más fácil de entender y de usar, y son menos propensos a ser afectados por los tipos indeseados de contenido generados de forma automática, aunque son susceptibles de ser falseados."
4214836368,The citation advantage of open‐access articles,"Abstract Four subjects—ecology, applied mathematics, sociology, and economics—were selected to assess whether there is a citation advantage between journal articles that have an open‐access (OA) version on the Internet compared to those articles that are exclusively toll access (TA). Citations were counted using the Web of Science, and the OA status of articles was determined by searching OAIster, OpenDOAR, Google, and Google Scholar. Of a sample of 4,633 articles examined, 2,280 (49%) were OA and had a mean citation count of 9.04 whereas the mean for TA articles was 5.76. There appears to be a clear citation advantage for those articles that are OA as opposed to those that are TA. This advantage, however, varies between disciplines, with sociology having the highest citation advantage, but the lowest number of OA articles, from the sample taken, and ecology having the highest individual citation count for OA articles, but the smallest citation advantage. Tests of correlation or association between OA status and a number of variables were generally found to weak or inconsistent. The cause of this citation advantage has not been determined."
2106336702,Pathfinder networks and author cocitation analysis: A remapping of paradigmatic information scientists,"Abstract In their 1998 article “Visualizing a discipline: An author cocitation analysis of information science, 1972–1995,” White and McCain used multidimensional scaling, hierarchical clustering, and factor analysis to display the specialty groupings of 120 highly‐cited (“paradigmatic”) information scientists. These statistical techniques are traditional in author cocitation analysis (ACA). It is shown here that a newer technique, Pathfinder Networks (PFNETs), has considerable advantages for ACA. In PFNETs, nodes represent authors, and explicit links represent weighted paths between nodes, the weights in this case being cocitation counts. The links can be drawn to exclude all but the single highest counts for author pairs, which reduces a network of authors to only the most salient relationships. When these are mapped, dominant authors can be defined as those with relatively many links to other authors (i.e., high degree centrality). Links between authors and dominant authors define specialties, and links between dominant authors connect specialties into a discipline. Maps are made with one rather than several computer routines and in one rather than many computer passes. Also, PFNETs can, and should, be generated from matrices of raw counts rather than Pearson correlations, which removes a computational step associated with traditional ACA. White and McCain's raw data from 1998 are remapped as a PFNET. It is shown that the specialty groupings correspond closely to those seen in the factor analysis of the 1998 article. Because PFNETs are fast to compute, they are used in AuthorLink, a new Web‐based system that creates live interfaces for cocited author retrieval on the fly."
1989318653,The Access/Impact Problem and the Green and Gold Roads to Open Access: An Update,"The research access/impact problem arises because journal articles are not accessible to all of their would-be users; hence, they are losing potential research impact. The solution is to make all articles open access (OA, i.e., accessible online, free for all). OA articles have significantly higher citation impact than non-OA articles. There are two roads to OA: the “golden” road (publish your article in an OA journal) and the “green” road (publish your article in a non-OA journal but also self-archive it in an OA archive). About 10% of journals are gold, but over 90% are already green (i.e., they have given their authors the green light to self-archive); yet only about 10–20% of articles have been self-archived. To reach 100% OA, self-archiving needs to be mandated by researchers’ employers and funders, as they are now increasingly beginning to do."
2997143876,"The academic, economic and societal impacts of Open Access: an evidence-based review","Ongoing debates surrounding Open Access to the scholarly literature are multifaceted and complicated by disparate and often polarised viewpoints from engaged stakeholders. At the current stage, Open Access has become such a global issue that it is critical for all involved in scholarly publishing, including policymakers, publishers, research funders, governments, learned societies, librarians, and academic communities, to be well-informed on the history, benefits, and pitfalls of Open Access. In spite of this, there is a general lack of consensus regarding the advantages or disadvantages of Open Access at multiple levels. This review aims to to be a resource for current knowledge on the impacts of Open Access by synthesizing important research in three major areas of impact: academic, economic and societal. While there is clearly much scope for additional research, several key trends are identified, including a broad citation advantage for researchers who publish openly, as well as additional benefits to the non-academic dissemination of their work. The economic case for Open Access is less well-understood, although it is clear that access to the research literature is key for innovative enterprises, and a range of governmental and non-governmental services. Furthermore, Open Access has the potential to save publishers and research funders considerable amounts of financial resources. The social case for Open Access is strong, in particular for advancing citizen science initiatives, and leveling the playing field for researchers in developing countries. Open Access supersedes all potential alternative modes of access to the scholarly literature through enabling unrestricted re-use, and long-term stability independent of financial constraints of traditional publishers that impede knowledge sharing. Open Access remains only one of the multiple challenges that the scholarly publishing system is currently facing. Yet, it provides one foundation for increasing engagement with researchers regarding ethical standards of publishing. We recommend that Open Access supporters focus their efforts on working to establish viable new models and systems of scholarly communication, rather than trying to undermine the existing ones as part of the natural evolution of the scholarly ecosystem. Based on this, future research should investigate the wider impacts of an ecosystem-wide transformation to a system of Open Research."
3122100588,How to normalize cooccurrence data? An analysis of some well‐known similarity measures,"Abstract In scientometric research, the use of cooccurrence data is very common. In many cases, a similarity measure is employed to normalize the data. However, there is no consensus among researchers on which similarity measure is most appropriate for normalization purposes. In this article, we theoretically analyze the properties of similarity measures for cooccurrence data, focusing in particular on four well‐known measures: the association strength, the cosine, the inclusion index, and the Jaccard index. We also study the behavior of these measures empirically. Our analysis reveals that there exist two fundamentally different types of similarity measures, namely, set‐theoretic measures and probabilistic measures. The association strength is a probabilistic measure, while the cosine, the inclusion index, and the Jaccard index are set‐theoretic measures. Both our theoretical and our empirical results indicate that cooccurrence data can best be normalized using a probabilistic measure. This provides strong support for the use of the association strength in scientometric research."
2106404266,Does Scientific Collaboration Increase the Impact of Ecological Articles?,"We examined the effects of different types of collaboration on the citation rates of 837 research papers published in Oecologia from 1998 through 2000. Multiauthored papers had higher annual citation rates, but also higher self-citation rates, than single-authored papers. Interdisciplinary collaboration between institutions increased citation rates, whereas in-house collaboration reduced them. Contrary to our predictions, international collaboration had no effect on the citation rates of ecological papers, and US ecologists benefited from collaboration more than their European colleagues. Altogether, our results indicate that scientific collaboration in ecology has a rather minor effect on the impact of the resulting publications, as measured by their citation rates."
2053031440,Effects of training on quality of peer review: randomised controlled trial,"Abstract Objective To determine the effects of training on the quality of peer review. Design Single blind randomised controlled trial with two intervention groups receiving different types of training plus a control group. Setting and participants Reviewers at a general medical journal. Interventions Attendance at a training workshop or reception of a self taught training package focusing on what editors want from reviewers and how to critically appraise randomised controlled trials. Main outcome measures Quality of reviews of three manuscripts sent to reviewers at four to six monthly intervals, evaluated using the validated review quality instrument; number of deliberate major errors identified; time taken to review the manuscripts; proportion recommending rejection of the manuscripts. Results Reviewers in the self taught group scored higher in review quality after training than did the control group (score 2.85 v 2.56; difference 0.29, 95% confidence interval 0.14 to 0.44; P = 0.001), but the difference was not of editorial significance and was not maintained in the long term. Both intervention groups identified significantly more major errors after training than did the control group (3.14 and 2.96 v 2.13; P < 0.001), and this remained significant after the reviewers' performance at baseline assessment was taken into account. The evidence for benefit of training was no longer apparent on further testing six months after the interventions. Training had no impact on the time taken to review the papers but was associated with an increased likelihood of recommending rejection (92% and 84% v 76%; P = 0.002). Conclusions Short training packages have only a slight impact on the quality of peer review. The value of longer interventions needs to be assessed."
2943934175,Ten Hot Topics around Scholarly Publishing,"The changing world of scholarly communication and the emerging new wave of ‘Open Science’ or ‘Open Research’ has brought to light a number of controversial and hotly debated topics. Evidence-based rational debate is regularly drowned out by misinformed or exaggerated rhetoric, which does not benefit the evolving system of scholarly communication. This article aims to provide a baseline evidence framework for ten of the most contested topics, in order to help frame and move forward discussions, practices, and policies. We address issues around preprints and scooping, the practice of copyright transfer, the function of peer review, predatory publishers, and the legitimacy of ‘global’ databases. These arguments and data will be a powerful tool against misinformation across wider academic research, policy and practice, and will inform changes within the rapidly evolving scholarly publishing system."
2090000005,The h‐index: a broad review of a new bibliometric indicator,"Purpose This review aims to show, broadly, how the h ‐index has become a subject of widespread debate, how it has spawned many variants and diverse applications since first introduced in 2005 and some of the issues in its use. Design/methodology/approach The review drew on a range of material published in 1990 or so sources published since 2005. From these sources, a number of themes were identified and discussed ranging from the h ‐index's advantages to which citation database might be selected for its calculation. Findings The analysis shows how the h ‐index has quickly established itself as a major subject of interest in the field of bibliometrics. Study of the index ranges from its mathematical underpinning to a range of variants perceived to address the indexes' shortcomings. The review illustrates how widely the index has been applied but also how care must be taken in its application. Originality/value The use of bibliometric indicators to measure research performance continues, with the h ‐index as its latest addition. The use of the h ‐index, its variants and many applications to which it has been put are still at the exploratory stage. The review shows the breadth and diversity of this research and the need to verify the veracity of the h ‐index by more studies."
3122426025,Does Co‐authorship Lead to Higher Academic Productivity?,"Abstract In recent decades, co‐authorship and policies aimed at inducing academic collaboration have increased simultaneously. Assuming that intellectual collaboration is exogenously determined, prior studies found a negative relationship between co‐authorship and productivity. I examine a panel data on economists publishing from 1970 to 2011 to test the causal effect of intellectual collaboration on intellectual output. As characteristics of the individual and her opportunity set are endogenously related to both collaboration and productivity, I instrument the amount of co‐authorship by the common research interest between an author and her potential co‐authors. After controlling for endogenous co‐authorship formation, unobservable heterogeneity and time varying factors, the effect of intellectual collaboration on individual performance becomes positive."
2953107155,A comparison of two techniques for bibliometric mapping: Multidimensional scaling and VOS,"Abstract VOS is a new mapping technique that can serve as an alternative to the well‐known technique of multidimensional scaling (MDS). We present an extensive comparison between the use of MDS and the use of VOS for constructing bibliometric maps. In our theoretical analysis, we show the mathematical relation between the two techniques. In our empirical analysis, we use the techniques for constructing maps of authors, journals, and keywords. Two commonly used approaches to bibliometric mapping, both based on MDS, turn out to produce maps that suffer from artifacts. Maps constructed using VOS turn out not to have this problem. We conclude that in general maps constructed using VOS provide a more satisfactory representation of a dataset than maps constructed using well‐known MDS approaches."
2128132605,Diffusion of scientific credits and the ranking of scientists,"Recently, the abundance of digital data is enabling the implementation of graph-based ranking algorithms that provide system level analysis for ranking publications and authors. Here, we take advantage of the entire Physical Review publication archive (1893-2006) to construct authors' networks where weighted edges, as measured from opportunely normalized citation counts, define a proxy for the mechanism of scientific credit transfer. On this network, we define a ranking method based on a diffusion algorithm that mimics the spreading of scientific credits on the network. We compare the results obtained with our algorithm with those obtained by local measures such as the citation count and provide a statistical analysis of the assignment of major career awards in the area of physics. A website where the algorithm is made available to perform customized rank analysis can be found at the address http://www.physauthorsrank.org."
2523550674,Professional and citizen bibliometrics: complementarities and ambivalences in the development and use of indicators—a state-of-the-art report,"Bibliometric indicators such as journal impact factors, h-indices, and total citation counts are algorithmic artifacts that can be used in research evaluation and management. These artifacts have no meaning by themselves, but receive their meaning from attributions in institutional practices. We distinguish four main stakeholders in these practices: (1) producers of bibliometric data and indicators; (2) bibliometricians who develop and test indicators; (3) research managers who apply the indicators; and (4) the scientists being evaluated with potentially competing career interests. These different positions may lead to different and sometimes conflicting perspectives on the meaning and value of the indicators. The indicators can thus be considered as boundary objects which are socially constructed in translations among these perspectives. This paper proposes an analytical clarification by listing an informed set of (sometimes unsolved) problems in bibliometrics which can also shed light on the tension between simple but invalid indicators that are widely used (e.g., the h-index) and more sophisticated indicators that are not used or cannot be used in evaluation practices because they are not transparent for users, cannot be calculated, or are difficult to interpret."
4238684574,A multi-disciplinary perspective on emergent and future innovations in peer review,"Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure? Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research. With the advent of web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review. We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation. Ideally, any new systems will demonstrate that they out-perform and reduce the biases of existing models as much as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages. We also propose a novel hybrid platform model that could, at least partially, resolve many of the socio-technical issues associated with peer review, and potentially disrupt the entire scholarly communication system. Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments."
1994490219,Introducing 'productive interactions' in social impact assessment,"Journal Article Introducing ‘productive interactions’ in social impact assessment Get access Jack Spaapen, Jack Spaapen Search for other works by this author on: Oxford Academic Google Scholar Leonie van Drooge Leonie van Drooge Search for other works by this author on: Oxford Academic Google Scholar Research Evaluation, Volume 20, Issue 3, September 2011, Pages 211–218, https://doi.org/10.3152/095820211X12941371876742 Published: 01 September 2011"
1987707043,Firm action needed on predatory journals,"They’re harming researchers in low and middle income countries most, but everyone must fight back"
2272387771,Nine Facts about Top Journals in Economics,"How has publishing in top economics journals changed since 1970? Using a data set that combines information on all articles published in the top-five journals from 1970 to 2012 with their Google Scholar citations, we identify nine key trends. First, annual submissions to the top-five journals nearly doubled from 1990 to 2012. Second, the total number of articles published in these journals actually declined from 400 per year in the late 1970s to 300 per year most recently. As a result, the acceptance rate has fallen from 15 percent to 6 percent, with potential implications for the career progression of young scholars. Third, one journal, the American Economic Review, now accounts for 40 percent of top-five publications, up from 25 percent in the 1970s. Fourth, recently published papers are on average three times longer than they were in the 1970s, contributing to the relative shortage of journal space. Fifth, the number of authors per paper has increased from 1.3 in 1970 to 2.3 in 2012, partly offsetting the fall in the number of articles per year. Sixth, citations for top-five publications are high: among papers published in the late 1990s, the median number of Google Scholar citations is 200. Seventh, the ranking of journals by citations has remained relatively stable, with the notable exception of the Quarterly Journal of Economics, which climbed from fourth place to first place over the past three decades. Eighth, citation counts are significantly higher for longer papers and those written by more coauthors. Ninth, although the fraction of articles from different fields published in the top five has remained relatively stable, there are important cohort trends in the citations received by papers from different fields, with rising citations to more recent papers in Development and International, and declining citations to recent papers in Econometrics and Theory. (JEL A14)"
3001491100,Software tools for conducting bibliometric analysis in science: An up-to-date review,"Bibliometrics has become an essential tool for assessing and analyzing the output of scientists, cooperation between universities, the effect of state-owned science funding on national research and development performance and educational efficiency, among other applications. Therefore, professionals and scientists need a range of theoretical and practical tools to measure experimental data. This review aims to provide an up-to-date review of the various tools available for conducting bibliometric and scientometric analyses, including the sources of data acquisition, performance analysis and visualization tools. The included tools were divided into three categories: general bibliometric and performance analysis, science mapping analysis, and libraries; a description of all of them is provided. A comparative analysis of the database sources support, pre-processing capabilities, analysis and visualization options were also provided in order to facilitate its understanding. Although there are numerous bibliometric databases to obtain data for bibliometric and scientometric analysis, they have been developed for a different purpose. The number of exportable records is between 500 and 50,000 and the coverage of the different science fields is unequal in each database. Concerning the analyzed tools, Bibliometrix contains the more extensive set of techniques and suitable for practitioners through Biblioshiny. VOSviewer has a fantastic visualization and is capable of loading and exporting information from many sources. SciMAT is the tool with a powerful pre-processing and export capability. In views of the variability of features, the users need to decide the desired analysis output and chose the option that better fits into their aims."
2088858100,Tweets vs. Mendeley readers: How do these two social media metrics differ?,"A set of 1.4 million biomedical papers was analyzed with regards to how often articles are mentioned on Twitter or saved by users on Mendeley. While Twitter is a microblogging platform used by a general audience to distribute information, Mendeley is a reference manager targeted at an academic user group to organize scholarly literature. Both platforms are used as sources for so-called altmetrics to measure a new kind of research impact. This analysis shows in how far they differ and compare to traditional citation impact metrics based on a large set of PubMed papers."
2115825262,Interdisciplinarity in science: A tentative typology of disciplines and research areas,"Abstract Interdisciplinarity is considered the best way to face practical research topics since synergy between traditional disciplines has proved very fruitful. Studies on interdisciplinarity from all possible perspectives are increasingly demanded. Different interdisciplinarity measures have been used in case studies but, up to now, no general interdisciplinarity indicator useful for Science Policy purposes has been accepted. The bibliometric methodology presented here provides a general overview of all scientific disciplines, with special attention to their interrelation. This work aims to establish a tentative typology of disciplines and research areas according to their degree of interdisciplinarity. Interdisciplinarity is measured through a series of indicators based on Institute for Scientific Information (ISI) multi‐assignation of journals in subject categories. Research areas and categories are described according to the quantity of their links (number of related categories) and their quality (with close or distant categories, diversity, and strength of links). High levels of interrelations between categories are observed. Four different types of categories are found through cluster analysis. This differentiates “big” interdisciplinarity, which links distant categories, from “small” interdisciplinarity, in which close categories are related. The location of specific categories in the clusters is discussed."
2108881982,"Assessment, evaluations, and definitions of research impact: A review","This article aims to explore what is understood by the term 'research impact' and to provide a comprehensive assimilation of available literature and information, drawing on global experiences to understand the potential for methods and frameworks of impact assessment being implemented for UK impact assessment. We take a more focused look at the impact component of the UK Research Excellence Framework taking place in 2014 and some of the challenges to evaluating impact and the role that systems might play in the future for capturing the links between research and impact and the requirements we have for these systems."
2145535121,A Comparison between Two Main Academic Literature Collections: Web of Science and Scopus Databases,"Nowadays, the world’s scientific community has been publishing an enormous number of papers in different scientific fields. In such environment, it is essential to know which databases are equally efficient and objective for literature searches. It seems that two most extensive databases are Web of Science and Scopus. Besides searching the literature, these two databases used to rank journals in terms of their productivity and the total citations received to indicate the journals impact, prestige or influence. This article attempts to provide a comprehensive comparison of these databases to answer frequent questions which researchers ask, such as: How Web of Science and Scopus are different? In which aspects these two databases are similar? Or, if the researchers are forced to choose one of them, which one should they prefer? For answering these questions, these two databases will be compared based on their qualitative and quantitative characteristics."
1496121555,New developments related to the Hirsch index,"It is shown that the h-index on the one hand, and the A- and g-indices on the other, measure different things. The A-index, however, seems overly sensitive to one extremely highly cited article. For this reason it would seem that the g-index is the more useful of the two. As to the h- and the g-index: they do measure different aspects of a scientist’s publication list. Certainly the h-index does not tell the full story, and, although a more sensitive indicator than the h-index, neither does the g-index. Taken together, g and h present a concise picture of a scientist’s achievements in terms of publications and citations."
2949208154,How are new citation‐based journal indicators adding to the bibliometric toolbox?,"Abstract The launching of Scopus and Google Scholar, and methodological developments in social‐network analysis have made many more indicators for evaluating journals available than the traditional impact factor, cited half‐life, and immediacy index of the ISI. In this study, these new indicators are compared with one another and with the older ones. Do the various indicators measure new dimensions of the citation networks, or are they highly correlated among themselves? Are they robust and relatively stable over time? Two main dimensions are distinguished—size and impact—which together shape influence. The h‐index combines the two dimensions and can also be considered as an indicator of reach (like Indegree). PageRank is mainly an indicator of size, but has important interactions with centrality measures. The Scimago Journal Ranking (SJR) indicator provides an alternative to the journal impact factor, but the computation is less easy."
2034165193,Measuring the Quality of Editorial Peer Review,"The quality of a process can only be tested against its agreed objectives. Editorial peer-review is widely used, yet there appears to be little agreement about how to measure its effects or processes.To identify outcome measures used to assess editorial peer review as performed by biomedical journals, we analyzed studies identified from 2 systematic reviews that measured the effects of editorial peer review on the quality of the output (ie, published articles) or of the process itself (eg, reviewers' comments).Ten studies used a variety of instruments to assess the quality of articles that had undergone peer review. Only 1, nonrandomized study compared the quality of articles published in peer-reviewed and non-peer-reviewed journals. The others measured the effects of variations in the peer-review process or used a before-and-after design to measure the effects of standard peer review on accepted articles. Eighteen studies measured the quality of reviewers' reports under different conditions such as blinding or after training. One study compared the time and cost of different review processes.Until we have properly defined the objectives of peer-review, it will remain almost impossible to assess or improve its effectiveness. The research needed to understand the broader effects of peer review poses many methodologic problems and would require the cooperation of many parts of the scientific community."
2109346842,Economic Implications of Alternative Scholarly Publishing Models Exploring the costs and benefits,"The aim of this project was to examine the costs and benefits of three alternative models for scholarly publishing - subscription publishing, open access publishing and self-archiving. The project involved two major phases: • Phase I: Identification of costs and benefits – sought to describe the three models of scholarly publishing, identify all the dimensions of cost and benefit for each of the models, and examine which of the main players in the scholarly communication system would be affected and how they would be affected; • Phase II: Quantification of costs and benefits – sought, where possible, to quantify the costs and benefits identified; identify and where possible quantify the cost and benefit implications for each of the main players in the scholarly communication system; and, where possible, compare the costs and benefits of the three models. While wide-ranging in scope, an important focus for the work was the implications of the three publishing models for UK higher education and fo r scholarly journal and book publishing – although other forms of publication and other stakeholders are included in the analysis."
1971871165,Scientific Collaboration Results in Higher Citation Rates of Published Articles,"Study Objective. The primary objective was to analyze the relationship between the citation rate of an article and the extent of collaboration. The secondary objective was to analyze the relationship between the number of authors/article and the number of institutions/article for the period of study. Methods. We counted the number of original research articles published in six leading journals—Cell, Science, Nature, New England Journal of Medicine, The Lancet, and Journal of the American Medical Association—for the years 1975, 1985, and 1995. For each article, we determined the number of authors and the number of separate institutions. We also determined the number of times each article that was published in 1995 was cited in future scientific articles from the Science Citation Index database. Results. Science, Cell, Nature, New England Journal of Medicine, The Lancet, and Journal of the American Medical Association had 2014, 868, 3856, 643, 785, and 465 total articles published/3-year study period, respectively. There was a median of 2, 2, 2, 3, 3, and 3 institutions/article, respectively. All of the final models had a significant linear author component for which all of the parameter estimates were positive, yet variable. Thus, the number of times an article was cited correlated significantly with the number of authors and the number of institutions. Conclusion. A correlation exists between the number of authors and the number of times an article is cited in other articles. Investigators who are open to collaborations and those who seem to adequately manage those collaborations produce a superior product that results in a higher impact."
4243763775,"Citation counting, citation ranking, and h‐index of human‐computer interaction researchers: A comparison of Scopus and Web of Science","Abstract This study examines the differences between Scopus and Web of Science in the citation counting, citation ranking, and h ‐index of 22 top human‐computer interaction (HCI) researchers from EQUATOR—a large British Interdisciplinary Research Collaboration project. Results indicate that Scopus provides significantly more coverage of HCI literature than Web of Science, primarily due to coverage of relevant ACM and IEEE peer‐reviewed conference proceedings. No significant differences exist between the two databases if citations in journals only are compared. Although broader coverage of the literature does not significantly alter the relative citation ranking of individual researchers, Scopus helps distinguish between the researchers in a more nuanced fashion than Web of Science in both citation counting and h ‐index. Scopus also generates significantly different maps of citation networks of individual scholars than those generated by Web of Science. The study also presents a comparison of h ‐index scores based on Google Scholar with those based on the union of Scopus and Web of Science. The study concludes that Scopus can be used as a sole data source for citation‐based research and evaluation in HCI, especially when citations in conference proceedings are sought, and that researchers should manually calculate h scores instead of relying on system calculations."
1029599486,The Continuing Growth of Global Cooperation Networks in Research: A Conundrum for National Governments,"Global collaboration continues to grow as a share of all scientific cooperation, measured as coauthorships of peer-reviewed, published papers. The percent of all scientific papers that are internationally coauthored has more than doubled in 20 years, and they account for all the growth in output among the scientifically advanced countries. Emerging countries, particularly China, have increased their participation in global science, in part by doubling their spending on R&d; they are increasingly likely to appear as partners on internationally coauthored scientific papers. Given the growth of connections at the international level, it is helpful to examine the phenomenon as a communications network and to consider the network as a new organization on the world stage that adds to and complements national systems. When examined as interconnections across the globe over two decades, a global network has grown denser but not more clustered, meaning there are many more connections but they are not grouping into exclusive 'cliques'. This suggests that power relationships are not reproducing those of the political system. The network has features an open system, attracting productive scientists to participate in international projects. National governments could gain efficiencies and influence by developing policies and strategies designed to maximize network benefits-a model different from those designed for national systems."
349374990,Publish or Perish,"Are you applying for tenure, promotion or a new job? Is your work cited in journals which are not ISI listed? Publish or Perish is designed to help individual academics to present their case for research impact to its best advantage."
2002297348,Harmonic Allocation of Authorship Credit: Source-Level Correction of Bibliometric Bias Assures Accurate Publication and Citation Analysis,"Authorship credit for multi-authored scientific publications is routinely allocated either by issuing full publication credit repeatedly to all coauthors, or by dividing one credit equally among all coauthors. The ensuing inflationary and equalizing biases distort derived bibliometric measures of merit by systematically benefiting secondary authors at the expense of primary authors. Here I show how harmonic counting, which allocates credit according to authorship rank and the number of coauthors, provides simultaneous source-level correction for both biases as well as accommodating further decoding of byline information. I also demonstrate large and erratic effects of counting bias on the original h-index, and show how the harmonic version of the h-index provides unbiased bibliometric ranking of scientific merit while retaining the original's essential simplicity, transparency and intended fairness. Harmonic decoding of byline information resolves the conundrum of authorship credit allocation by providing a simple recipe for source-level correction of inflationary and equalizing bias. Harmonic counting could also offer unrivalled accuracy in automated assessments of scientific productivity, impact and achievement."
2059145052,A bibliometric classificatory approach for the study and assessment of research performance at the individual level: The effects of age on productivity and impact,"The authors set forth a general methodology for conducting bibliometric analyses at the micro level. It combines several indicators grouped into three factors or dimensions, which characterize different aspects of scientific performance. Different profiles or “classes” of scientists are described according to their research performance in each dimension. A series of results based on the findings from the application of this methodology to the study of Spanish National Research Council scientists in Spain in three thematic areas are presented. Special emphasis is made on the identification and description of top scientists from structural and bibliometric perspectives. The effects of age on the productivity and impact of the different classes of scientists are analyzed. The classificatory approach proposed herein may prove a useful tool in support of research assessment at the individual level and for exploring potential determinants of research success. © 2010 Wiley Periodicals, Inc."
1979943489,Gender Differences in Publication Output: Towards an Unbiased Metric of Research Performance,"We examined the publication records of a cohort of 168 life scientists in the field of ecology and evolutionary biology to assess gender differences in research performance. Clear discrepancies in publication rate between men and women appear very early in their careers and this has consequences for the subsequent citation of their work. We show that a recently proposed index designed to rank scientists fairly is in fact strongly biased against female researchers, and advocate a modified index to assess men and women on a more equitable basis."
2120262090,How fractional counting of citations affects the impact factor: Normalization in terms of differences in citation potentials among fields of science,"The Impact Factors (IFs) of the Institute for Scientific Information suffer from a number of drawbacks, among them the statistics—Why should one use the mean and not the median?—and the incomparability among fields of science because of systematic differences in citation behavior among fields. Can these drawbacks be counteracted by fractionally counting citation weights instead of using whole numbers in the numerators? (a) Fractional citation counts are normalized in terms of the citing sources and thus would take into account differences in citation behavior among fields of science. (b) Differences in the resulting distributions can be tested statistically for their significance at different levels of aggregation. (c) Fractional counting can be generalized to any document set including journals or groups of journals, and thus the significance of differences among both small and large sets can be tested. A list of fractionally counted IFs for 2008 is available online at http:www.leydesdorff.net/weighted_if/weighted_if.xls The between-group variance among the 13 fields of science identified in the U.S. Science and Engineering Indicators is no longer statistically significant after this normalization. Although citation behavior differs largely between disciplines, the reflection of these differences in fractionally counted citation distributions can not be used as a reliable instrument for the classification."
2064816492,Altmetric: enriching scholarly content with article‐level discussion and metrics,"ABSTRACT Scholarly content is increasingly being discussed, shared, and bookmarked online by researchers. Altmetric is a start‐up that focuses on tracking, collecting, and measuring this activity on behalf of publishers; here we describe our approach and general philosophy. Over the past year we have seen sharing and discussion activity around approximately 750,000 articles. The average number of articles shared each day grows by 5–10% a month. We look at examples of how people are interacting with papers online and at how publishers can collect and present the resulting data to deliver real value to their authors and readers."
2158156550,The ups and downs of peer review,"This article traces the history of peer review of scientific publications, plotting the development of the process from its inception to its present-day application. We discuss the merits of peer review and its weaknesses, both perceived and real, as well as the practicalities of several major proposed changes to the system. It is our hope that readers will gain a better appreciation of the complexities of the process and, when serving as reviewers themselves, will do so in a manner that will enhance the utility of the exercise. We also propose the development of an international on-line training program for accreditation of potential referees."
2104070376,Causes for the Persistence of Impact Factor Mania,"ABSTRACT Numerous essays have addressed the misuse of the journal impact factor for judging the value of science, but the practice continues, primarily as a result of the actions of scientists themselves. This seemingly irrational behavior is referred to as ""impact factor mania."" Although the literature on the impact factor is extensive, little has been written on the underlying causes of impact factor mania. In this perspective, we consider the reasons for the persistence of impact factor mania and its pernicious effects on science. We conclude that impact factor mania persists because it confers significant benefits to individual scientists and journals. Impact factor mania is a variation of the economic theory known as the ""tragedy of the commons,"" in which scientists act rationally in their own self-interests despite the detrimental consequences of their actions on the overall scientific enterprise. Various measures to reduce the influence of the impact factor are considered. IMPORTANCE Science and scientists are currently afflicted by an epidemic of mania manifested by associating the value of research with the journal where the work is published rather than the content of the work itself. The mania is causing profound distortions in the way science is done that are deleterious to the overall scientific enterprise. In this essay, we consider the forces responsible for the persistence of the mania and conclude that it is maintained because it disproportionately benefits elements of the scientific enterprise, including certain well-established scientists, journals, and administrative interests. Our essay suggests steps that can be taken to deal with this debilitating and destructive epidemic."
4239797392,Toward a consensus map of science,"Abstract A consensus map of science is generated from an analysis of 20 existing maps of science. These 20 maps occur in three basic forms: hierarchical, centric, and noncentric (or circular). The consensus map, generated from consensus edges that occur in at least half of the input maps, emerges in a circular form. The ordering of areas is as follows: mathematics is (arbitrarily) placed at the top of the circle, and is followed clockwise by physics, physical chemistry, engineering, chemistry, earth sciences, biology, biochemistry, infectious diseases, medicine, health services, brain research, psychology, humanities, social sciences, and computer science. The link between computer science and mathematics completes the circle. If the lowest weighted edges are pruned from this consensus circular map, a hierarchical map stretching from mathematics to social sciences results. The circular map of science is found to have a high level of correspondence with the 20 existing maps, and has a variety of advantages over hierarchical and centric forms. A one‐dimensional Riemannian version of the consensus map is also proposed."
3046545064,The Science of Science,"The rapid development of digital libraries and the proliferation of scholarly big data have created an unprecedented opportunity to explore scientific production and reward at scale. Fueled by the data exploration and computational advances in digital libraries, the science of science is an emerging multidisciplinary field that aims to quantify patterns for scientific relationships and dependencies, and how scientific progress emerges from the scholarly big data. In this tutorial, we will provide an overview of the science of science, including major topics on scientific careers, scientific collaborations and scientific ideas. We will also discuss its historical context, the state-of-art models and exciting discoveries, and promising future directions for participants interested in mining scholarly big data."
3021454583,The limitations to our understanding of peer review,"Abstract Peer review is embedded in the core of our knowledge generation systems, perceived as a method for establishing quality or scholarly legitimacy for research, while also often distributing academic prestige and standing on individuals. Despite its critical importance, it curiously remains poorly understood in a number of dimensions. In order to address this, we have analysed peer review to assess where the major gaps in our theoretical and empirical understanding of it lie. We identify core themes including editorial responsibility, the subjectivity and bias of reviewers, the function and quality of peer review, and the social and epistemic implications of peer review. The high-priority gaps are focused around increased accountability and justification in decision-making processes for editors and developing a deeper, empirical understanding of the social impact of peer review. Addressing this at the bare minimum will require the design of a consensus for a minimal set of standards for what constitutes peer review, and the development of a shared data infrastructure to support this. Such a field requires sustained funding and commitment from publishers and research funders, who both have a commitment to uphold the integrity of the published scholarly record. We use this to present a guide for the future of peer review, and the development of a new research discipline based on the study of peer review."
1515302379,On the Opportunities and Limitations of the H-index,A new indicator for the assessment of the research performance of individual scientists was suggested by Jorge E. Hirsch in 2005. Several opportunities and limitations of this new measure are discussed.
3104836124,Men Set Their Own Cites High: Gender and Self-citation across Fields and over Time,"How common is self-citation in scholarly publication, and does the practice vary by gender? Using novel methods and a data set of 1.5 million research papers in the scholarly database JSTOR published between 1779 and 2011, the authors find that nearly 10 percent of references are self-citations by a paper’s authors. The findings also show that between 1779 and 2011, men cited their own papers 56 percent more than did women. In the last two decades of data, men self-cited 70 percent more than women. Women are also more than 10 percentage points more likely than men to not cite their own previous work at all. While these patterns could result from differences in the number of papers that men and women authors have published rather than gender-specific patterns of self-citation behavior, this gender gap in self-citation rates has remained stable over the last 50 years, despite increased representation of women in academia. The authors break down self-citation patterns by academic field and number of authors and comment on potential mechanisms behind these observations. These findings have important implications for scholarly visibility and cumulative advantage in academic careers."
1904486621,Mendeley readership altmetrics for medical articles: An analysis of 45 fields,"Medical research is highly funded and often expensive and so is particularly important to evaluate effectively. Nevertheless, citation counts may accrue too slowly for use in some formal and informal evaluations. It is therefore important to investigate whether alternative metrics could be used as substitutes. This article assesses whether one such altmetric, M endeley readership counts, correlates strongly with citation counts across all medical fields, whether the relationship is stronger if student readers are excluded, and whether they are distributed similarly to citation counts. Based on a sample of 332,975 articles from 2009 in 45 medical fields in Scopus, citation counts correlated strongly (about 0.7; 78% of articles had at least one reader) with Mendeley readership counts (from the new version 1 applications programming interface [ API ]) in almost all fields, with one minor exception, and the correlations tended to decrease slightly when student readers were excluded. Readership followed either a lognormal or a hooked power law distribution, whereas citations always followed a hooked power law, showing that the two may have underlying differences."
2131837970,How can we investigate citation behavior? A study of reasons for citing literature in communication,"Authors' motivations for citing documents are addressed through a literature review and an empirical study. Replicating an investigation in psychology, the works of two highly-cited authors in the discipline of communication were identified, and all of the authors who cited them during the period 1995–1997 were surveyed. The instrument posed 32 questions about why a certain document was cited, plus questions about the citer's relationship to the cited author and document. Most findings were similar to the psychology study, including a tendency to cite ""concept markers"" representing a genre of work. Authors in communication were more likely to have an interpersonal connection to cited authors, and to cite literature reviews—their most common reason for citation. Three types of judgments about cited works were found to best predict citation: (1) that the work was novel, well-known, and a concept-marker; (2) that citing it might promote the authority of one's own work; and (3) that the work deserved criticism. Suggestions are made for further research, especially regarding the anomalous role of creativity in cited works."
2072233627,Gender Differences in Productivity,"Since 1984, when Cole and Zuckerman referred to gender differences in productivity among academic scientists as a puzzle, sociologists have sought to explain these differences by incorporating primarily institutional-level factors. In addition to these factors, the author contends that an undertheorized and heretofore unmeasured concept—the extent of research specialization—can also help explain the process by which gender affects research productivity. Although some researchers have examined areas of specialization, the extent of research specialization has been completely neglected in studies of academic careers. Using a probability sample of academics in two disciplines (sociology and linguistics), primary data collection, and simultaneous equation modeling, the author finds that the extent of research specialization is a critical intervening variable: Women specialize less than men and thereby lose out on an important means of increasing their productivity."
2168595914,Scientific and technical human capital: an alternative model for research evaluation,"We provide an alternative model for evaluating science and technology projects and programs. Our approach, a ""scientific and technical human capital"" (S&T human capital) model, gives less attention to the discrete products and immediate outcomes from scientific projects and programs - the usual focus of evaluations - and more attention to scientists' career trajectories and their sustained ability to contribute and enhance their capabilities. S&T human capital encompasses not only the individual human capital endowments but also researchers' tacit knowledge, craft knowledge, and know-how. S&T human capital further includes the social capital that scientists continually draw upon in creating knowledge - for knowledge creation is neither a solitary nor singular event. In sum, it is this expanded notion of human capital when paired with a productive social capital network that enables researchers to create and transform knowledge and ideas in ways that would not be possible without these resources. We review literature contributing to an S&T human capital model and consider some of the practical data and measurement issues entailed in implementing such an approach."
2037997493,Collective credit allocation in science,"Significance The increasing dominance of multiauthor papers is straining the credit system of science: although for single-author papers, the credit is obvious and undivided, for multiauthor papers, credit assignment varies from discipline to discipline. Consequently, each research field runs its own informal credit allocation system, which is hard to decode for outsiders. Here we develop a discipline-independent algorithm to decipher the collective credit allocation process within science, capturing each coauthor’s perceived contribution to a publication. The proposed method provides scientists and policy-makers an effective tool to quantify and compare the scientific contribution of each researcher without requiring familiarity with the credit allocation system of the specific discipline."
2106469901,"The Decision-Making Constraints and Processes of Grant Peer Review, and Their Effects on the Review Outcome","When distributing grants, research councils use peer expertise as a guarantee for supporting the best projects. However, there are no clear norms for assessments, and there may be a large variation in what criteria reviewers emphasize - and how they are emphasized. The determinants of peer review may therefore be accidental, in the sense that who reviews what research and how reviews are organized may determine outcomes. This paper deals with how the review process affects the outcome of grant review. The case study considers the procedures of The Research Council of Norway, which practises several different grant-review models, and consequently is especially suited for explorations of the implications of different models. Data sources are direct observation of panel meetings, interviews with panel members and study of applications and review documents. A central finding is that rating scales and budget restrictions are more important than review guidelines for the kind of criteria applied by the reviewers. The decision-making methods applied by the review panels when ranking proposals are found to have substantial effects on the outcome. Some ranking methods tend to support uncontroversial and safe projects, whereas other methods give better chances for scholarly pluralism and controversial research."
2794944797,"Assessing scientists for hiring, promotion, and tenure","Assessment of researchers is necessary for decisions of hiring, promotion, and tenure. A burgeoning number of scientific leaders believe the current system of faculty incentives and rewards is misaligned with the needs of society and disconnected from the evidence about the causes of the reproducibility crisis and suboptimal quality of the scientific publication record. To address this issue, particularly for the clinical and life sciences, we convened a 22-member expert panel workshop in Washington, DC, in January 2017. Twenty-two academic leaders, funders, and scientists participated in the meeting. As background for the meeting, we completed a selective literature review of 22 key documents critiquing the current incentive system. From each document, we extracted how the authors perceived the problems of assessing science and scientists, the unintended consequences of maintaining the status quo for assessing scientists, and details of their proposed solutions. The resulting table was used as a seed for participant discussion. This resulted in six principles for assessing scientists and associated research and policy implications. We hope the content of this paper will serve as a basis for establishing best practices and redesigning the current approaches to assessing scientists by the many players involved in that process."
2110562193,Bibliometrics to webometrics,"Bibliometrics has changed out of all recognition since 1958; becoming established as a field, being taught widely in library and information science schools, and being at the core of a number of science evaluation research groups around the world. This was all made possible by the work of Eugene Garfield and his Science Citation Index. This article reviews the distance that bibliometrics has travelled since 1958 by comparing early bibliometrics with current practice, and by giving an overview of a range of recent developments, such as patent analysis, national research evaluation exercises, visualization techniques, new applications, online citation indexes, and the creation of digital libraries. Webometrics, a modern, fast-growing offshoot of bibliometrics, is reviewed in detail. Finally, future prospects are discussed with regard to both bibliometrics and webometrics."
2940242975,"Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations","We analyzed how often and in what ways the Journal Impact Factor (JIF) is currently used in review, promotion, and tenure (RPT) documents of a representative sample of universities from the United States and Canada. 40% of research-intensive institutions and 18% of master’s institutions mentioned the JIF, or closely related terms. Of the institutions that mentioned the JIF, 87% supported its use in at least one of their RPT documents, 13% expressed caution about its use, and none heavily criticized it or prohibited its use. Furthermore, 63% of institutions that mentioned the JIF associated the metric with quality, 40% with impact, importance, or significance, and 20% with prestige, reputation, or status. We conclude that use of the JIF is encouraged in RPT evaluations, especially at research-intensive universities, and that there is work to be done to avoid the potential misuse of metrics like the JIF."
2007567350,The Academic Caste System: Prestige Hierarchies in PhD Exchange Networks,"The prestige of academic departments is commonly understood as rooted in the scholarly productivity of their faculty and graduates. I use the theories of Weber and Bourdieu to advance an alternative view of departmental prestige, which I show is an effect a department's position within networks of association and social exchange—that is, it is a form of social capital. The social network created by the exchange of PhDs among departments is the most important network of this kind. Using data on the exchange of PhDs among sociology departments, I apply network analysis to investigate this alternative conception of departmental prestige and to demonstrate its superiority over the conventional view. Within sociology, centrality within interdepartmental hiring networks explains 84 percent of the variance in departmental prestige. Similar findings are reported for history and political science. This alternative understanding of academic prestige helps clarify anomalies—e.g., the variance in prestige unconnected to scholarly productivity, the strong association between department size and prestige, and the long-term stability of prestige rankings—encountered in research that is based on the more conventional view."
4230129368,The Access/Impact Problem and the Green and Gold Roads to Open Access,"AbstractAbstractThe research access/impact problem arises because journal articles are not accessible to all of their would-be users; hence, they are losing potential research impact. The solution is to make all articles Open Access (OA; i.e., accessible online, free for all). OA articles have significantly higher citation impact than non-OA articles. There are two roads to OA: the “golden” road (publish your article in an OA journal) and the “green” road (publish your article in a non-OA journal but also self-archive it in an OA archive). Only 5% of journals are gold, but over 90% are already green (i.e., they have given their authors the green light to self-archive); yet only about 10–20% of articles have been self-archived. To reach 100% OA, self-archiving needs to be mandated by researchers' employers and funders, as the United Kingdom and the United States have recently recommended, and universities need to implement that mandate."
2150340219,Beyond the Durfee square: Enhancing the h-index to score total publication output,"An individual's h-index corresponds to the number h of his/her papers that each has at least h citations. When the citation count of an article exceeds h, however, as is the case for the hundreds or even thousands of citations that accompany the most highly cited papers, no additional credit is given (these citations falling outside the so-called ""Durfee square""). We propose a new bibliometric index, the ""tapered h-index"" (h T), that positively enumerates all citations, yet scoring them on an equitable basis with h. The career progression of h T and h are compared for six eminent scientists in contrasting fields. Calculated h T for year 2006 ranged between 44.32 and 72.03, with a corresponding range in h of 26 to 44. We argue that the h T-index is superior to h, both theoretically (it scores all citations), and because it shows smooth increases from year to year as compared with the irregular jumps seen in h. Conversely, the original h-index has the benefit of being conceptually easy to visualise. Qualitatively, the two indices show remarkable similarity (they are closely correlated), such that either can be applied with confidence."
363554780,"Interdisciplinarity and Impact: Distinct Effects of Variety, Balance, and Disparity","Interdisciplinary research is increasingly recognized as the solution to today’s challenging scientific and societal problems, but the relationship between interdisciplinary research and scientific impact is still unclear. This paper studies the association between the degree of interdisciplinarity and the number of citations at the paper level. Different from previous studies compositing various aspects of interdisciplinarity into a single indicator, we use factor analysis to uncover distinct dimensions of interdisciplinarity corresponding to variety, balance, and disparity. We estimate Poisson models with journal fixed effects and robust standard errors to analyze the divergent relationships between these three factors and citations. We find that long-term (13-year) citations (1) increase at an increasing rate with variety, (2) decrease with balance, and (3) increase at a decreasing rate with disparity. Furthermore, interdisciplinarity also affects the process of citation accumulation: (1) although variety and disparity have positive effects on long-term citations, they have negative effects on short-term (3-year) citations, and (2) although balance has a negative effect on long-term citations, its negative effect is insignificant in the short run. These findings have important implications for interdisciplinary research and science policy."
4240573486,A bibliometric classificatory approach for the study and assessment of research performance at the individual level: The effects of age on productivity and impact,"Abstract The authors set forth a general methodology for conducting bibliometric analyses at the micro level. It combines several indicators grouped into three factors or dimensions, which characterize different aspects of scientific performance. Different profiles or “classes” of scientists are described according to their research performance in each dimension. A series of results based on the findings from the application of this methodology to the study of Spanish National Research Council scientists in Spain in three thematic areas are presented. Special emphasis is made on the identification and description of top scientists from structural and bibliometric perspectives. The effects of age on the productivity and impact of the different classes of scientists are analyzed. The classificatory approach proposed herein may prove a useful tool in support of research assessment at the individual level and for exploring potential determinants of research success."
2102017123,Systematic inequality and hierarchy in faculty hiring networks,"The faculty job market plays a fundamental role in shaping research priorities, educational outcomes, and career trajectories among scientists and institutions. However, a quantitative understanding of faculty hiring as a system is lacking. Using a simple technique to extract the institutional prestige ranking that best explains an observed faculty hiring network-who hires whose graduates as faculty-we present and analyze comprehensive placement data on nearly 19,000 regular faculty in three disparate disciplines. Across disciplines, we find that faculty hiring follows a common and steeply hierarchical structure that reflects profound social inequality. Furthermore, doctoral prestige alone better predicts ultimate placement than a U.S. News & World Report rank, women generally place worse than men, and increased institutional prestige leads to increased faculty production, better faculty placement, and a more influential position within the discipline. These results advance our ability to quantify the influence of prestige in academia and shed new light on the academic system."
2132218857,Google Scholar: The New Generation of Citation Indexes,"Google Scholar (http://scholar.google.com) provides a new method of locating potentially relevant articles on a given subject by identifying subsequent articles that cite a previously published article. An important feature of Google Scholar is that researchers can use it to trace interconnections among authors citing articles on the same topic and to determine the frequency with which others cite a specific article, as it has a “cited by” feature. This study begins with an overview of how to use Google Scholar for citation analysis and identifies advanced search techniques not well documented by Google Scholar. This study also compares the citation counts provided by Web of Science and Google Scholar for articles in the field of “Webometrics.” It makes several suggestions for improving Google Scholar. Finally, it concludes that Google Scholar provides a free alternative or complement to other citation indexes."
2150607630,World citation and collaboration networks: uncovering the role of geography in science,"Modern information and communication technologies, especially the Internet, have diminished the role of spatial distances and territorial boundaries on the access and transmissibility of information. This has enabled scientists for closer collaboration and internationalization. Nevertheless, geography remains an important factor affecting the dynamics of science. Here we present a systematic analysis of citation and collaboration networks between cities and countries, by assigning papers to the geographic locations of their authors' affiliations. The citation flows as well as the collaboration strengths between cities decrease with the distance between them and follow gravity laws. In addition, the total research impact of a country grows linearly with the amount of national funding for research & development. However, the average impact reveals a peculiar threshold effect: the scientific output of a country may reach an impact larger than the world average only if the country invests more than about 100,000 USD per researcher annually."
3122634626,"Looking Across and Looking Beyond the Knowledge Frontier: Intellectual Distance, Novelty, and Resource Allocation in Science","Selecting among alternative projects is a core management task in all innovating organizations. In this paper, we focus on the evaluation of frontier scientific research projects. We argue that the “intellectual distance” between the knowledge embodied in research proposals and an evaluator’s own expertise systematically relates to the evaluations given. To estimate relationships, we designed and executed a grant proposal process at a leading research university in which we randomized the assignment of evaluators and proposals to generate 2,130 evaluator–proposal pairs. We find that evaluators systematically give lower scores to research proposals that are closer to their own areas of expertise and to those that are highly novel. The patterns are consistent with biases associated with boundedly rational evaluation of new ideas. The patterns are inconsistent with intellectual distance simply contributing “noise” or being associated with private interests of evaluators. We discuss implications for policy, managerial intervention, and allocation of resources in the ongoing accumulation of scientific knowledge. This paper was accepted by Lee Fleming, entrepreneurship and innovation."
2395804017,From Sole Investigator to Team Scientist: Trends in the Practice and Study of Research Collaboration,"This article reviews trends in the practice and study of research collaboration, focusing on journal publications in academic science. I briefly describe the different styles and types of collaboration and then focus on the drivers of the trend toward increased collaboration and on its consequences for both individual researchers and science more generally. Scholarship on collaboration seems partial to delineating its benefits; this review highlights the increasing body of research that focuses instead on the possible costs of collaboration. The synthesis reveals several topics that are ripe for investigation, including the impact of collaboration on the contributing authors and their work, the use of multiple methods and measures, and research integrity. I applaud a few recent efforts to overcome the perennial file-drawer problem by gaining access to collaborations that do not result in publication and thus are typically removed from public review and the research analyst's eye."
2006187383,Determinants of research citation impact in nanoscience and nanotechnology,"This study investigates a range of metrics available when a nanoscience and nanotechnology article is published to see which metrics correlate more with the number of citations to the article. It also introduces the degree of internationality of journals and references as new metrics for this purpose. The journal impact factor; the impact of references; the internationality of authors, journals, and references; and the number of authors, institutions, and references were all calculated for papers published in nanoscience and nanotechnology journals in the W eb of S cience from 2007 to 2009. Using a zero‐inflated negative binomial regression model on the data set, the impact factor of the publishing journal and the citation impact of the cited references were found to be the most effective determinants of citation counts in all four time periods. In the entire 2007 to 2009 period, apart from journal internationality and author numbers and internationality, all other predictor variables had significant effects on citation counts."
2173130871,Web indicators for research evaluation. Part 2: Social media metrics,"Esta revisión bibliográfica evalúa indicadores derivados de medios sociales, tanto generales como académicos. Tales indicadores han sido llamados, influmétricos, altmétricos, métricas de medios sociales, o tipo de webmetría. Recientemente los han comercializado algunas empresas y los emplean algunos editores y administradores universitarios. Las métricas de medios sociales analizados aquí se derivan principalmente de Twitter, Facebook, Google+, F1000, Mendeley, ResearchGate y Academia.edu. Tienen el aparente potencial de ofrecer indicadores rápidos y gratuitos del impacto social de la investigación, o de impactos académicos de un tipo diferente, que complementan los indicadores obtenidos de los tradicionales índices de citas. Aunque no es prudente emplearlos en las evaluaciones formales de personas e instituciones, debido a que pueden ser falseados fácilmente y a la falta de evidencia real de que reflejen fielmente el impacto de la investigación en la sociedad, son útiles para las evaluaciones experimentales y para investigar la ciencia misma. Los conteos de lectores de Mendeley son particularmente prometedores."
2061451796,Not just a matter of time: Field differences and the shaping of electronic media in supporting scientific communication,"The shift towards the use of electronic media in scholarly communication appears to be an inescapable imperative. However, these shifts are uneven, both with respect to field and with respect to the form of communication. Different scientific fields have developed and use distinctly different communicative forums, both in the paper and electronic arenas, and these forums play different communicative roles within the field. One common claim is that we are in the early stages of an electronic revolution, that it is only a matter of time before other fields catch up with the early adopters, and that all fields converge on a stable set of electronic forums. A social shaping of technology (SST) perspective helps us to identify important social forces—centered around disciplinary constructions of trust and of legitimate communication—that pull against convergence. This analysis concludes that communicative plurality and communicative heterogeneity are durable features of the scholarly landscape, and that we are likely to see field differences in the use of and meaning ascribed to communications forums persist, even as overall use of electronic communications technologies both in science and in society as a whole increases."
2614298075,Social media use in the research workflow,"ABSTRACT The paper reports on a major international survey, covering 2,000 researchers, which investigated the use of social media in the research workflow. The topic is the second to emerge from the Charleston Observatory, the research adjunct of the popular annual Charleston Conference ( http://www.katina.info/conference/ ). The study shows that social media have found serious application at all points of the research lifecycle, from identifying research opportunities to disseminating findings at the end. The three most popular social media tools in a research setting were those for collaborative authoring, conferencing, and scheduling meetings. The most popular brands used tend to be mainstream anchor technologies or ‘household brands‘, such as Twitter. Age is a poor predictor of social media use in a research context, and humanities and social science scholars avail themselves most of social media. Journals, conference proceedings, and edited books remain the core traditional means of disseminating research, with institutional repositories highly valued as well, but social media have become important complementary channels for disseminating and discovering research."
1887814245,Prominent but Less Productive,"Federal agencies and universities in the U.S. promote interdisciplinary research because it presumably spurs transformative, innovative science. Using data on almost 900 research-center–based scientists and their 32,000 published articles, along with a set of unpublished papers, we assess whether such research is indeed beneficial and whether costs accompany the potential benefits. Existing research highlights this tension: whereas the innovation literature suggests that spanning disciplines is beneficial because it allows scientists to see connections across fields, the categories literature suggests that spanning disciplines is penalized because the resulting research may be lower quality or confusing to place. To investigate this, we empirically distinguish production and reception effects and highlight a new production penalty: lower productivity, which may be attributable to cognitive and collaborative challenges associated with interdisciplinary research and/or hurdles in the review process. Using an innovative measure of interdisciplinary research that considers the similarity of the disciplines spanned, we document both penalties (fewer papers published) and benefits (increased citations) associated with it and show that it is a high-risk, high-reward endeavor, one that partly depends on field-level interdisciplinarity."
2150455238,"Users, narcissism and control – tracking the impact of scholarly publications in the 21st century","This report explores the explosion of tracking tools that have accompanied the surge of web based information instruments. Is it possible to monitor ‘real-time’ how new research findings are being read, cited, used and transformed in practical results and applications? And what are the potential risks and disadvantages of the new tracking tools? This report aims to contribute to a better understanding of these developments by providing a detailed assessment of the currently available novel tools and methodologies. A total of 16 quite different tools are assessed. What is the scientific and social impact of my research publications? This question has been of interest to scientists and scholars since the inception of modern science 400 years ago. But it was hard to answer. This may now be changing. Scholarship is transforming into a variety of digital networked forms. These developments have created new possibilities and challenges in the evaluation of the quality of research. This is of interest to research funders assessing the quality of research. It is also relevant to the individual researchers interested in assessing their career development."
1993171516,"To share the fame in a fair way,hmmodifieshfor multi-authored manuscripts","The h-index has been introduced by Hirsch as a useful measure to characterize the scientific output of a researcher. I suggest a simple modification in order to take multiple co-authorship appropriately into account, by counting each paper only fractionally according to (the inverse of) the number of authors. The resulting hm-indices for eight famous physicists lead to a different ranking from the original h-indices."
2130900574,The most influential journals: Impact Factor and Eigenfactor,"The sequential production of cell types during neural development is controlled by temporal identity transcription factors, and heterochronic expression of these factors in progenitors reprograms developmental potential and promotes the ...Temporal identity factors are sufficient to reprogram developmental competence of neural progenitors and shift cell fate output, but whether they can also reprogram the identity of terminally differentiated cells is unknown. To address this question, we ..."
4250137533,"Accuracy and completeness of publication and citation records in the Web of Science, PsycINFO, and Google Scholar: A case study for the computation of h indices in Psychology","Abstract Hirsch's h index is becoming the standard measure of an individual's research accomplishments. The aggregation of individuals' measures is also the basis for global measures at institutional or national levels. To investigate whether the h index can be reliably computed through alternative sources of citation records, the Web of Science (WoS), PsycINFO and Google Scholar (GS) were used to collect citation records for known publications of four Spanish psychologists. Compared with WoS, PsycINFO included a larger percentage of publication records, whereas GS outperformed WoS and PsycINFO in this respect. Compared with WoS, PsycINFO retrieved a larger number of citations in unique areas of psychology, but it retrieved a smaller number of citations in areas that are close to statistics or the neurosciences, whereas GS retrieved the largest numbers of citations in all cases. Incorrect citations were scarce in Wos (0.3%), more prevalent in PsycINFO (1.1%), and overwhelming in GS (16.5%). All platforms retrieved unique citations, the largest set coming from GS. WoS and PsycINFO cover distinct areas of psychology unevenly, thus applying different penalties on the h index of researches working in different fields. Obtaining fair and accurate h indices required the union of citations retrieved by all three platforms."
2462819058,A simple proposal for the publication of journal citation distributions,"Abstract Although the Journal Impact Factor (JIF) is widely acknowledged to be a poor indicator of the quality of individual papers, it is used routinely to evaluate research and researchers. Here, we present a simple method for generating the citation distributions that underlie JIFs. Application of this straightforward protocol reveals the full extent of the skew of these distributions and the variation in citations received by published papers that is characteristic of all scientific journals. Although there are differences among journals across the spectrum of JIFs, the citation distributions overlap extensively, demonstrating that the citation performance of individual papers cannot be inferred from the JIF. We propose that this methodology be adopted by all journals as a move to greater transparency, one that should help to refocus attention on individual pieces of work and counter the inappropriate usage of JIFs during the process of research assessment."
2127765089,Supporting and Enhancing Scholarship in the Digital Age: The Role of Open-Access Institutional Repositories,"Scholarly communication and publishing are increasingly taking place in the electronic environment. With a growing proportion of the scholarly record now existing only in digital format, serious and pressing issues regarding access and preservation are being raised that are central to future scholarship. At the same time, the desire of scholars to maximize readership of their research and to take control of the scholarly communication process back from the restrictive domain of commercial publishing has prompted the proliferation of access options and experimental models of publishing. This paper examines the emerging trend of university-based institutional repositories (IRs) designed to capture the scholarly output of an institution and to maximize the research impact of this output. The relationship of this trend to the open access movement is discussed and challenges and opportunities for using IRs to promote new modes of scholarship are provided."
4233744313,Google Scholar citations and Google Web/URL citations: A multi‐discipline exploratory analysis,"Abstract We use a new data gathering method, “Web/URL citation,” Web/URL and Google Scholar to compare traditional and Web‐based citation patterns across multiple disciplines (biology, chemistry, physics, computing, sociology, economics, psychology, and education) based upon a sample of 1,650 articles from 108 open access (OA) journals published in 2001. A Web/URL citation of an online journal article is a Web mention of its title, URL, or both. For each discipline, except psychology, we found significant correlations between Thomson Scientific (formerly Thomson ISI, here: ISI) citations and both Google Scholar and Google Web/URL citations. Google Scholar citations correlated more highly with ISI citations than did Google Web/URL citations, indicating that the Web/URL method measures a broader type of citation phenomenon. Google Scholar citations were more numerous than ISI citations in computer science and the four social science disciplines, suggesting that Google Scholar is more comprehensive for social sciences and perhaps also when conference articles are valued and published online. We also found large disciplinary differences in the percentage overlap between ISI and Google Scholar citation sources. Finally, although we found many significant trends, there were also numerous exceptions, suggesting that replacing traditional citation sources with the Web or Google Scholar for research impact calculations would be problematic."
2032160096,Peer reviews and bibliometric indicators: a comparative study at a Norwegian university,"Journal Article Peer reviews and bibliometric indicators: a comparative study at a Norwegian university Get access Dag W Aksnes, Dag W Aksnes Search for other works by this author on: Oxford Academic Google Scholar Randi Elisabeth Taxt Randi Elisabeth Taxt Search for other works by this author on: Oxford Academic Google Scholar Research Evaluation, Volume 13, Issue 1, April 2004, Pages 33–41, https://doi.org/10.3152/147154404781776563 Published: 01 April 2004"
2271411371,The 100 classic papers of orthopaedic surgery,"The credibility and creativity of an author may be gauged by the number of scientific papers he or she has published, as well as the frequency of citations of a particular paper reflecting the impact of the data on the area of practice. The object of this study was to identify and analyse the qualities of the top 100 cited papers in orthopaedic surgery. The database of the Science Citation Index of the Institute for Scientific Information (1945 to 2008) was used. A total of 1490 papers were cited more than 100 times, with the top 100 being subjected to further analysis. The majority originated in the United States, followed by the United Kingdom. The top 100 papers were published in seven specific orthopaedic journals. Analysis of the most-cited orthopaedic papers allows us a unique insight into the qualities, characteristics and clinical innovations required for a paper to attain 'classic' status."
1992037052,Ranking scientific publications using a model of network traffic,"To account for strong ageing characteristics of citation networks, we modify the PageRank algorithm by initially distributing random surfers exponentially with age, in favour of more recent publications. The output of this algorithm, which we call CiteRank, is interpreted as approximate traffic to individual publications in a simple model of how researchers find new information. We optimize parameters of our algorithm to achieve the best performance. The results are compared for two rather different citation networks: all American Physical Society publications between 1893 and 2003 and the set of high-energy physics theory (hep-th) preprints. Despite major differences between these two networks, we find that their optimal parameters for the CiteRank algorithm are remarkably similar. The advantages and performance of CiteRank over more conventional methods of ranking publications are discussed."
1971909015,Mapping the network of global science: comparing international co-authorships from 1990 to 2000,"Science and technology has been shown to result from dynamic interactions occurring at many levels and across various sectors. This study maps and analyses the network operating at the global level. Using the Science Citation Index (CD-Rom version) for 1990 and 2000, the networks created by international co-authorships are revealed. We analyse the observed linkages at the global level and on regional bases. The architecture of the network is further explored using statistical methods and factor analysis to reveal intense relationships as well as the core members of a global network. Findings show that, in the ten years between 1990 and 2000, the global network has expanded to include more nations, but the structure of this network has become more pronounced. Regional networks show emerging hubs. However, in some cases, the regions have been disconnected from the global level. A core set of advanced industrial nations has expanded from six in 1990 to eight in 2000. Factor analysis suggests that various large countries compete with each other for partners in the global network."
2129156960,Women in Economics: Moving Up or Falling Off the Academic Career Ladder?,"The percentage of economics doctorates awarded to women has increased over the past twenty years. This article considers whether women Ph.D. economists have increased their representation in academia, particularly at higher tenured ranks. Our study draws upon several empirical approaches and multiple data sets for the 1990s. We find that when compared with other academic disciplines, women in economics are less likely to get tenure and take longer to achieve it. Although gender differences in productivity and the effect of children on promotion partly explain women's lesser chances of receiving tenure in economics, a significant portion of the gender promotion gap remains unexplained by observable characteristics."
2009918649,The intellectual structure of the strategic management field: an author co‐citation analysis,"Abstract This paper complements a recent study by Ramos‐Rodriguez and Ruiz‐Navarro (2004) that investigated the intellectual structure of the strategic management field through co‐citation analysis. By using authors as the units of analysis and incorporating all the citations that are included in the Science Citation Index and the Social Science Citation Index, we trace the evolution of the intellectual structure of the strategic management field during the period 1980–2000. Using a variety of data analytic techniques such as multidimensional scaling, factor analysis, and Pathfinder analysis, we (1) delineate the subfields that constitute the intellectual structure of strategic management; (2) determine the relationships between the subfields; (3) identify authors who play a pivotal role in bridging two or more conceptual domains of research; and (4) graphically map the intellectual structure in two‐dimensional space in order to visualize spatial distances between intellectual themes. The analysis provides insights about the influence of individual authors as well as changes in their influence over time. Copyright © 2007 John Wiley & Sons, Ltd."
2886105060,Gendered Citation Patterns across Political Science and Social Science Methodology Fields,"Accumulated evidence identifies discernible gender gaps across many dimensions of professional academic careers including salaries, publication rates, journal placement, career progress, and academic service. Recent work in political science also reveals gender gaps in citations, with articles written by men citing work by other male scholars more often than work by female scholars. This study estimates the gender gap in citations across political science subfields and across methodological subfields within political science, sociology, and economics. The research design captures variance across research areas in terms of the underlying distribution of female scholars. We expect that subfields within political science and social science disciplines with more women will have smaller gender citation gaps, a reduction of the “Matthew effect” where men’s research is viewed as the most central and important in a field. However, gender citation gaps may persist if a “Matilda effect” occurs whereby women’s research is viewed as less important or their ideas are attributed to male scholars, even as a field becomes more diverse. Analysing all articles published from 2007–2016 in several journals, we find that female scholars are significantly more likely than mixed gender or male author teams to cite research by their female peers, but that these citation rates vary depending on the overall distribution of women in their field. More gender diverse subfields and disciplines produce smaller gender citation gaps, consistent with a reduction in the “Matthew effect”. However, we also observe undercitation of work by women, even in journals that publish mostly female authors. While improvements in gender diversity in academia increase the visibility and impact of scholarly work by women, implicit biases in citation practices in the social sciences persist."
3121240669,"Citations In Economics: Measurement, Uses, and Impacts","I describe and compare sources of data on citations in economics and the statistics derived from them. Constructing data sets of the post-publication citation histories of articles published in the “top five” journals in the 1970s and 2000s, I examine distributions and life cycles of citations, compare citation histories of articles in different subspecialties in economics, and present evidence on the history and heterogeneity of those journals' impacts and the marginal citation productivity of additional coauthors. I use a new data set of the lifetime citation histories of over 1,000 economists from thirty universities to rank economics departments by various measures and demonstrate the importance of intra- and interdepartmental heterogeneity in productivity. Throughout, the discussion summarizes earlier work, including the impacts of citations on salaries and nonmonetary rewards, and how citations reflect judgments about research quality in economics and the importance of economic ideas. (JEL A14, I23)"
2806145641,Why do authors publish in predatory journals?,"This study examines the reasons why authors publish in ‘predatory’ OA journals. In total, 50 journals were randomly selected from Beall's list of ‘predatory’ journals. Different methods, including WHOIS tracking, were utilized to query basic information about the selected journals, including location and registrant. Then, 300 articles were randomly selected from within selected journals in various scientific fields. Authors of the selected articles were contacted and sent survey questions to complete. A grounded theory qualitative methods approach was used for data collection and analysis. The results demonstrated that most of these journals were located in the developing world, usually Asia or Africa, even when they claimed they were in the USA or UK . Furthermore, four themes emerged after authors’ survey responses were coded, categorized, and sub‐categorized. The themes were: social identity threat, unawareness, high pressure, and lack of research proficiency . Scholars in the developing world felt that reputable Western journals might be prejudiced against them and sometimes felt more comfortable publishing in journals from the developing world. Other scholars were unaware of the reputation of the journals in which they published and would not have selected them had they known. However, some scholars said they would still have published in the same journals if their institution recognised them. The pressure to ‘publish or perish’ was another factor influencing many scholars’ decisions to publish in these fast‐turnaround journals. In some cases, researchers did not have adequate guidance and felt they lacked the knowledge of research to submit to a more reputable journal. More needs to be done by institutions and reputable journals to make researchers aware of the problem of ‘predatory’ journals."
2951610038,On the relationship between interdisciplinarity and scientific impact,"Abstract This article analyzes the effect of interdisciplinarity on the scientific impact of individual articles. Using all the articles published in Web of Science in 2000, we define the degree of interdisciplinarity of a given article as the percentage of its cited references made to journals of other disciplines. We show that although for all disciplines combined there is no clear correlation between the level of interdisciplinarity of articles and their citation rates, there are nonetheless some disciplines in which a higher level of interdisciplinarity is related to a higher citation rates. For other disciplines, citations decline as interdisciplinarity grows. One characteristic is visible in all disciplines: Highly disciplinary and highly interdisciplinary articles have a low scientific impact. This suggests that there might be an optimum of interdisciplinarity beyond which the research is too dispersed to find its niche and under which it is too mainstream to have high impact. Finally, the relationship between interdisciplinarity and scientific impact is highly determined by the citation characteristics of the disciplines involved: Articles citing citation‐intensive disciplines are more likely to be cited by those disciplines and, hence, obtain higher citation scores than would articles citing non‐citation‐intensive disciplines."
2952391320,Content‐based and algorithmic classifications of journals: Perspectives on the dynamics of scientific communication and indexer effects,"Abstract The aggregated journal‐journal citation matrix—based on the Journal Citation Reports (JCR) of the Science Citation Index—can be decomposed by indexers or algorithmically. In this study, we test the results of two recently available algorithms for the decomposition of large matrices against two content‐based classifications of journals: the ISI Subject Categories and the field/subfield classification of Glänzel and Schubert (2003). The content‐based schemes allow for the attribution of more than a single category to a journal, whereas the algorithms maximize the ratio of within‐category citations over between‐category citations in the aggregated category‐category citation matrix. By adding categories, indexers generate between‐category citations, which may enrich the database, for example, in the case of inter‐disciplinary developments. Algorithmic decompositions, on the other hand, are more heavily skewed towards a relatively small number of categories, while this is deliberately counter‐acted upon in the case of content‐based classifications. Because of the indexer effects, science policy studies and the sociology of science should be careful when using content‐based classifications, which are made for bibliographic disclosure, and not for the purpose of analyzing latent structures in scientific communications. Despite the large differences among them, the four classification schemes enable us to generate surprisingly similar maps of science at the global level. Erroneous classifications are cancelled as noise at the aggregate level, but may disturb the evaluation locally."
2110462941,F1000 Recommendations as a Potential New Data Source for Research Evaluation: A Comparison With Citations,"F 1000 is a postpublication peer review service for biological and medical research. F 1000 recommends important publications in the biomedical literature, and from this perspective F 1000 could be an interesting tool for research evaluation. By linking the complete database of F 1000 recommendations to the Web of Science bibliographic database, we are able to make a comprehensive comparison between F 1000 recommendations and citations. We find that about 2% of the publications in the biomedical literature receive at least one F 1000 recommendation. Recommended publications on average receive 1.30 recommendations, and more than 90% of the recommendations are given within half a year after a publication has appeared. There turns out to be a clear correlation between F 1000 recommendations and citations. However, the correlation is relatively weak, at least weaker than the correlation between journal impact and citations. More research is needed to identify the main reasons for differences between recommendations and citations in assessing the impact of publications."
3121279336,Economics: An Emerging Small World,"We study the evolution of social distance among economists over the period 1970–2000. While the number of economists has more than doubled, the distance between them, which was already small, has declined significantly. The key to understanding the short average distances is the observation that economics is spanned by a collection of interlinked stars. A star is an economist who writes with many other economists, most of whom have few coauthors and generally do not write with each other."
2070382614,The “total cost of publication” in a hybrid open‐access environment: Institutional approaches to funding journal article‐processing charges in combination with subscriptions,"As open‐access ( OA ) publishing funded by article‐processing charges ( APCs ) becomes more widely accepted, academic institutions need to be aware of the “total cost of publication” (TCP), comprising subscription costs plus APCs and additional administration costs. This study analyzes data from 23 UK institutions covering the period 2007–2014 modeling the TCP . It shows a clear rise in centrally managed APC payments from 2012 onward, with payments projected to increase further. As well as evidencing the growing availability and acceptance of OA publishing, these trends reflect particular UK policy developments and funding arrangements intended to accelerate the move toward OA publishing (“ G old” OA ). Although the mean value of APCs has been relatively stable, there was considerable variation in APC prices paid by institutions since 2007. In particular, “hybrid” subscription/ OA journals were consistently more expensive than fully OA journals. Most APCs were paid to large “traditional” commercial publishers who also received considerable subscription income. New administrative costs reported by institutions varied considerably. The total cost of publication modeling shows that APCs are now a significant part of the TCP for academic institutions, in 2013 already constituting an average of 10% of the TCP (excluding administrative costs)."
2003844967,The impact of free access to the scientific literature: a review of recent research,"Objectives:The paper reviews recent studies that evaluate the impact of free access (open access) on the behavior of scientists as authors, readers, and citers in developed and developing nations. It also examines the extent to which the biomedical literature is used by the general public.Method:The paper is a critical review of the literature, with systematic description of key studies.Results:Researchers report that their access to the scientific literature is generally good and improving. For authors, the access status of a journal is not an important consideration when deciding where to publish. There is clear evidence that free access increases the number of article downloads, although its impact on article citations is not clear. Recent studies indicate that large citation advantages are simply artifacts of the failure to adequately control for confounding variables. The effect of free access on the general public's use of the primary medical literature has not been thoroughly evaluated.Conclusions:Recent studies provide little evidence to support the idea that there is a crisis in access to the scholarly literature. Further research is needed to investigate whether free access is making a difference in non-research contexts and to better understand the dissemination of scientific literature through peer-to-peer networks and other informal mechanisms."
2041540760,How and why scholars cite on Twitter,"Scholars are increasingly using the microblogging service Twitter as a communication platform. Since citing is a central practice of scholarly communication, we investigated whether and how scholars cite on Twitter. We conducted interviews and harvested 46,515 tweets from a sample of 28 scholars and found that they do cite on Twitter, though often indirectly. Twitter citations are part of a fast-moving conversation that participants believe reflects scholarly impact. Twitter citation metrics could augment traditional citation analysis, supporting a ""scientometrics 2.0."""
1554939242,On the robustness of the h‐index,"Abstract The h ‐index (Hirsch, 2005) is robust, remaining relatively unaffected by errors in the long tails of the citations‐rank distribution, such as typographic errors that short‐change frequently cited articles and create bogus additional records. This robustness, and the ease with which h ‐indices can be verified, support the use of a Hirsch‐type index over alternatives such as the journal impact factor. These merits of the h ‐index apply both to individuals and to journals."
2063709399,"Deflated, inflated and phantom citation counts","Purpose The purpose of this paper is to clarify some issues regarding citation indexing, analysis and searching. Design/methodology/approach The paper begins with a discussion on an article in the D‐Lib Magazine and then focuses on deflated citation counts and inflated and phantom citation counts. Findings The combination of the inflated citation count values dispensed by Google Scholar (GS) with the ignorance and shallowness of some GS enthusiasts can be a real mix for real scholars. Originality/value The paper offers insight into deflated, inflated and phantom citation counts."
2011443643,Open access and Scopus: A new approach to scientific visibility from the standpoint of access,"The last few years have seen the emergence of several open access (OA) options in scholarly communication, which can be grouped broadly into two areas referred to as gold and green roads. Several recent studies have shown how large the extent of OA is, but there have been few studies showing the impact of OA in the visibility of journals covering all scientific fields and geographical regions. This research presents a series of informative analyses providing a broad overview of the degree of proliferation of OA journals in a data sample of about 17,000 active journals indexed in Scopus. This study shows a new approach to scientific visibility from a systematic combination of four databases: Scopus, the Directory of Open Access Journals, Rights Metadata for Open Archiving"
2104192199,Emerging trends in regenerative medicine: a scientometric analysis inCiteSpace,"Introduction: Regenerative medicine involves research in a number of fields and disciplines such as stem cell research, tissue engineering and biological therapy in general. As research in these areas advances rapidly, it is critical to keep abreast of emerging trends and critical turns of the development of the collective knowledge."
2028083052,Measuring the effectiveness of scientific gatekeeping,"Significance Peer review is an institution of enormous importance for the careers of scientists and the content of published science. The decisions of gatekeepers—editors and peer reviewers—legitimize scientific findings, distribute professional rewards, and influence future research. However, appropriate data to gauge the quality of gatekeeper decision-making in science has rarely been made publicly available. Our research tracks the popularity of rejected and accepted manuscripts at three elite medical journals. We found that editors and reviewers generally made good decisions regarding which manuscripts to promote and reject. However, many highly cited articles were surprisingly rejected. Our research suggests that evaluative strategies that increase the mean quality of published science may also increase the risk of rejecting unconventional or outstanding work."
1602102908,Content‐based citation analysis: The next generation of citation analysis,"Traditional citation analysis has been widely applied to detect patterns of scientific collaboration, map the landscapes of scholarly disciplines, assess the impact of research outputs, and observe knowledge transfer across domains. It is, however, limited, as it assumes all citations are of similar value and weights each equally. Content‐based citation analysis ( CCA ) addresses a citation's value by interpreting each one based on its context at both the syntactic and semantic levels. This paper provides a comprehensive overview of CAA research in terms of its theoretical foundations, methodical approaches, and example applications. In addition, we highlight how increased computational capabilities and publicly available full‐text resources have opened this area of research to vast possibilities, which enable deeper citation analysis, more accurate citation prediction, and increased knowledge discovery."
2062091329,Unethical practices in authorship of scientific papers,"Abstract Over the past few decades, there has been an increase in the number of multi‐author papers within scientific journals. This increase, in combination with the pressure to publish within academia, has precipitated various unethical authorship practices within biomedical research. These include dilution of authorship responsibility, ‘guest’, ‘pressured’ and ‘ghost’ authorship, and obfuscation of authorship credit within by‐lines. Other authorship irregularities include divided and duplicate publication. This article discusses these problems and why the International Committee of Medical Journal Editors guidelines are failing to control them."
596883077,Beyond bibliometrics: harnessing multidimensional indicators of scholarly impact,"Bibliometrics has moved well beyond the mere tracking of bibliographic citations. The web enables new ways to measure scholarly productivity and impact, making available tools and data that can reveal patterns of intellectual activity and impact that were previously invisible: mentions, acknowledgments, endorsements, downloads, recommendations, blog posts, tweets. This book describes recent theoretical and practical advances in metrics-based research, examining a variety of alternative metrics -- or altmetrics -- while also considering the ethical and cultural consequences of relying on metrics to assess the quality of scholarship. Once the domain of information scientists and mathematicians, bibliometrics is now a fast-growing, multidisciplinary field that ranges from webometrics to scientometrics to influmetrics. The contributors to Beyond Bibliometrics discuss the changing environment of scholarly publishing, the effects of open access and Web 2.0 on genres of discourse, novel analytic methods, and the emergence of next-generation metrics in a performance-conscious age. ContributorsMayur Amin, Judit Bar-Ilan, Johann Bauer, Lutz Bornmann, Benjamin F. Bowman, Kevin W. Boyack, Blaise Cronin, Ronald Day, Nicola De Bellis, Jonathan Furner, Yves Gingras, Stefanie Haustein, Edwin Henneken, Peter A. Hook, Judith Kamalski, Richard Klavans, Kayvan Kousha, Michael Kurtz, Mark Largent, Julia Lane, Vincent Lariviere, Loet Leydesdorff, Werner Marx, Katherine W. McCain, Margit Palzenberger, Andrew Plume, Jason Priem, Rebecca Rosen, Hermann Schier, Hadas Shema, Cassidy R. Sugimoto, Mike Thelwall, Daril Vilhena, Jevin West, Paul Wouters"
2603021916,Gender bias in scholarly peer review,"Peer review is the cornerstone of scholarly publishing and it is essential that peer reviewers are appointed on the basis of their expertise alone. However, it is difficult to check for any bias in the peer-review process because the identity of peer reviewers generally remains confidential. Here, using public information about the identities of 9000 editors and 43000 reviewers from the Frontiers series of journals, we show that women are underrepresented in the peer-review process, that editors of both genders operate with substantial same-gender preference (homophily), and that the mechanisms of this homophily are gender-dependent. We also show that homophily will persist even if numerical parity between genders is reached, highlighting the need for increased efforts to combat subtler forms of gender bias in scholarly publishing."
1975107573,"Scholarly network similarities: How bibliographic coupling networks, citation networks, cocitation networks, topical networks, coauthorship networks, and coword networks relate to each other","This study explores the similarity among six types of scholarly networks aggregated at the institution level, including bibliographic coupling networks, citation networks, cocitation networks, topical networks, coauthorship networks, and coword networks. Cosine distance is chosen to measure the similarities among the six networks. The authors found that topical networks and coauthorship networks have the lowest similarity; cocitation networks and citation networks have high similarity; bibliographic coupling networks and cocitation networks have high similarity; and coword networks and topical networks have high similarity. In addition, through multidimensional scaling, two dimensions can be identified among the six networks: D imension 1 can be interpreted as citation‐based versus noncitation‐based, and D imension 2 can be interpreted as social versus cognitive. The authors recommend the use of hybrid or heterogeneous networks to study research interaction and scholarly communications."
1519117689,Beyond citations: Scholars' visibility on the social Web,"Traditionally, scholarly impact and visibility have been measured by counting publications and citations in the scholarly literature. However, increasingly scholars are also visible on the Web, establishing presences in a growing variety of social ecosystems. But how wide and established is this presence, and how do measures of social Web impact relate to their more traditional counterparts? To answer this, we sampled 57 presenters from the 2010 Leiden STI Conference, gathering publication and citations counts as well as data from the presenters' Web ""footprints."" We found Web presence widespread and diverse: 84% of scholars had homepages, 70% were on LinkedIn, 23% had public Google Scholar profiles, and 16% were on Twitter. For sampled scholars' publications, social reference manager bookmarks were compared to Scopus and Web of Science citations; we found that Mendeley covers more than 80% of sampled articles, and that Mendeley bookmarks are significantly correlated (r=.45) to Scopus citation counts."
2570427954,Big Scholarly Data: A Survey,"With the rapid growth of digital publishing, harvesting, managing, and analyzing scholarly information have become increasingly challenging. The term Big Scholarly Data is coined for the rapidly growing scholarly data, which contains information including millions of authors, papers, citations, figures, tables, as well as scholarly networks and digital libraries. Nowadays, various scholarly data can be easily accessed and powerful data analysis technologies are being developed, which enable us to look into science itself with a new perspective. In this paper, we examine the background and state of the art of big scholarly data. We first introduce the background of scholarly data management and relevant technologies. Second, we review data analysis methods, such as statistical analysis, social network analysis, and content analysis for dealing with big scholarly data. Finally, we look into representative research issues in this area, including scientific impact evaluation, academic recommendation, and expert finding. For each issue, the background, main challenges, and latest research are covered. These discussions aim to provide a comprehensive review of this emerging area. This survey paper concludes with a discussion of open issues and promising future directions."
2793772122,Publication patterns in the social sciences and humanities: evidence from eight European countries,"This study investigates patterns in the language and type of social sciences and humanities (SSH) publications in non-English speaking European countries to demonstrate that such patterns are related not only to discipline but also to each country’s cultural and historic heritage. We investigate publication patterns that occur across SSH publications of the whole of the SSH and of economics and business, law, and philosophy and theology publications in the Czech Republic, Denmark, Finland, Flanders (Belgium), Norway, Poland, Slovakia, and Slovenia. We use data from 74,022 peer-reviewed publications from 2014registered in at least one of the eight countries’ national databases and for 272,376 peer- reviewed publications from the period of 2011–2014 registered in at least one of the seven countries’ national databases (for all countries except Slovakia). Our findings show that publication patterns differ both between fields (e.g. patterns in law differ from those in economics and business in the same way in Flanders and Finland) and within fields (e.g. patterns in law in the Czech Republic differ from patterns in law in Finland). We observe that the publication patterns are stable and quite similar in West European and Nordic countries, whereas in Central and Eastern European countries the publication patterns demonstrate considerable changes. Nevertheless, in all countries, the share of articles and the share of publications in English is on the rise. We conclude with recommendations for science policy and highlight that internationalization policies in non-English speaking countries should consider various starting points and cultural heritages in different countries."
2116110543,Assessing the future landscape of scholarly communication: an exploration of faculty values and needs in seven disciplines,"Despite the increase in open access publishing peer-reviewed journals still hold sway in many academic disciplines according to this US study. While many studies have addressed specific issues like the costs of launching academic journals and the finances of university presses, there has not yet been a broad-scale, comprehensive analysis focusing on how the university and its various stakeholders, most notably faculty, value traditional and emerging forms of scholarly communication. The research focuses on understanding faculty needs and practices for in-progress scholarly communication as well as archival publication. Among the goals is providing a broader understanding of the full array of activities related to the scholarly communication lifecycle in order to enable the accurate assessment of the academy’s future communication and publication landscape. The following are among the questions driving the work: • What will scholars in various core disciplines want to do in their research and with their research results, and what new forms of communication do or do not support those needs? • How will scholars want to disseminate and receive input on their work at various stages of the scholarly communication lifecycle? • What are the emerging trends in research and publication practices? • What is the scope and depth of pent-up demand for new models of communication in various sectors/disciplines? • How do institutions and other stakeholders support these faculty needs, if at all? We suggest that more innovation does and will occur first during in-progress communication than in final, archival publication. One can foresee a scenario where useful and effective innovations in in-progress communication may eventually serve to drive improvements in final, archival publication. It is therefore worthwhile to gain deeper insight into the needs, motives, and new capabilities within in-progress communication as well as final, archival publication. We describe here our results based on the responses of 160 interviewees across 45, mostly elite, research institutions in seven selected academic fields: archaeology, astrophysics, biology, economics, history, music, and political science. The report is divided into eight chapters, which include a document synthesizing our methods and research results plus seven detailed disciplinary case studies."
1995893870,Can Mendeley bookmarks reflect readership? A survey of user motivations,"Although M endeley bookmarking counts appear to correlate moderately with conventional citation metrics, it is not known whether academic publications are bookmarked in M endeley in order to be read or not. Without this information, it is not possible to give a confident interpretation of altmetrics derived from Mendeley. In response, a survey of 860 M endeley users shows that it is reasonable to use M endeley bookmarking counts as an indication of readership because most (55%) users with a M endeley library had read or intended to read at least half of their bookmarked publications. This was true across all broad areas of scholarship except for the arts and humanities (42%). About 85% of the respondents also declared that they bookmarked articles in M endeley to cite them in their publications, but some also bookmark articles for use in professional (50%), teaching (25%), and educational activities (13%). Of course, it is likely that most readers do not record articles in M endeley and so these data do not represent all readers. In conclusion, M endeley bookmark counts seem to be indicators of readership leading to a combination of scholarly impact and wider professional impact."
153854902,Institutional Repositories: Evaluating the Reasons for Non-use of Cornell University's Installation of DSpace,"Problem: While there has been considerable attention dedicated to the development and implementation of institutional repositories, there has been little done to evaluate them, especially with regards to faculty participation. Purpose: This article reports on a three-part evaluative study of institutional repositories. We describe the contents and participation in Cornell’s DSpace and compare these results with seven university DSpace installations. Through in-depth interviews with eleven faculty members in the sciences, social sciences and humanities, we explore their attitudes, motivations, and behaviors for non-participation in institutional repositories. Results: Cornell’s DSpace is largely underpopulated and underused by its faculty. Many of its collections are empty, and most collections contain few items. Those collections that experience steady growth are collections in which the university has made an administrative investment, such are requiring deposits of theses and dissertations into DSpace. Cornell faculty have little knowledge of and little motivation to use DSpace. Many faculty use alternatives to institutional repositories, such as their personal Web pages and disciplinary repositories, which are perceived to have higher community salience than one’s affiliate institution. Faculty gave many reasons for not using repositories: redundancy with other modes of disseminating information, the learning curve, confusion with copyright, fear of plagiarism and having one’s work scooped, associating one’s work with inconsistent quality, and whether posting a manuscript constitutes “publishing”. Conclusion: While some librarians perceive a crisis in scholarly communication as a crisis in access to the literature, Cornell faculty perceive this essentially as a non-issue. Each discipline has a normative culture, largely defined by their reward system and traditions. If the goal of institutional repositories is to capture and preserve the scholarship of one’s faculty, institutional repositories will need to address this cultural diversity."
2036137014,Design and Update of a Classification System: The UCSD Map of Science,"Global maps of science can be used as a reference system to chart career trajectories, the location of emerging research frontiers, or the expertise profiles of institutes or nations. This paper details data preparation, analysis, and layout performed when designing and subsequently updating the UCSD map of science and classification system. The original classification and map use 7.2 million papers and their references from Elsevier's Scopus (about 15,000 source titles, 2001-2005) and Thomson Reuters' Web of Science (WoS) Science, Social Science, Arts & Humanities Citation Indexes (about 9,000 source titles, 2001-2004)-about 16,000 unique source titles. The updated map and classification adds six years (2005-2010) of WoS data and three years (2006-2008) from Scopus to the existing category structure-increasing the number of source titles to about 25,000. To our knowledge, this is the first time that a widely used map of science was updated. A comparison of the original 5-year and the new 10-year maps and classification system show (i) an increase in the total number of journals that can be mapped by 9,409 journals (social sciences had a 80% increase, humanities a 119% increase, medical (32%) and natural science (74%)), (ii) a simplification of the map by assigning all but five highly interdisciplinary journals to exactly one discipline, (iii) a more even distribution of journals over the 554 subdisciplines and 13 disciplines when calculating the coefficient of variation, and (iv) a better reflection of journal clusters when compared with paper-level citation data. When evaluating the map with a listing of desirable features for maps of science, the updated map is shown to have higher mapping accuracy, easier understandability as fewer journals are multiply classified, and higher usability for the generation of data overlays, among others."
2968900048,A standardized citation metrics author database annotated for scientific field,"Citation metrics are widely used and misused. We have created a publicly available database of 100,000 top scientists that provides standardized information on citations, h-index, coauthorship-adjusted hm-index, citations to papers in different authorship positions, and a composite indicator. Separate data are shown for career-long and single-year impact. Metrics with and without self-citations and ratio of citations to citing papers are given. Scientists are classified into 22 scientific fields and 176 subfields. Field- and subfield-specific percentiles are also provided for all scientists who have published at least five papers. Career-long data are updated to end of 2017 and to end of 2018 for comparison."
2591659376,The operationalization of “fields” as WoS subject categories (WCs) in evaluative bibliometrics: The cases of “library and information science” and “science & technology studies”,"Normalization of citation scores using reference sets based on W eb of S cience subject categories ( WC s) has become an established (“best”) practice in evaluative bibliometrics. For example, the T imes H igher E ducation W orld U niversity R ankings are, among other things, based on this operationalization. However, WC s were developed decades ago for the purpose of information retrieval and evolved incrementally with the database; the classification is machine‐based and partially manually corrected. Using the WC “information science & library science” and the WC s attributed to journals in the field of “science and technology studies,” we show that WC s do not provide sufficient analytical clarity to carry bibliometric normalization in evaluation practices because of “indexer effects.” Can the compliance with “best practices” be replaced with an ambition to develop “best possible practices”? New research questions can then be envisaged."
2022369531,Not by Productivity Alone: How Visibility and Specialization Contribute to Academic Earnings,"The popular adage “publish or perish” has long defined individual career strategies as well as scholarly investigations of earnings inequality in academe, as researchers have relied heavily on research productivity to explain earnings inequality among faculty members. Academia, however, has changed dramatically in the last few decades: it has become larger and more demographically diverse, and fears of overspecialization prompt calls for interdisciplinary approaches. In this new environment, other factors, in addition to productivity, are likely relevant to our understanding of earnings differentials. In this article, I assess whether two additional factors—visibility and the extent of research specialization—contribute to men's earning advantage. Using probability samples of tenure-track academics in two disciplines, a variety of data sources, and innovative measures, I find that both factors are highly relevant to the process by which earnings are determined. Women earn less than men largely because they specialize less. Lower levels of specialization hinder productivity, productivity enhances visibility, and visibility has a direct, positive, and significant effect on salary. I discuss the practical implications of these findings and lay the foundation for a broader theory of the role of research specialization in work processes."
3123554164,"Large-scale comparison of bibliographic data sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic","Abstract We present a large-scale comparison of five multidisciplinary bibliographic data sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. The comparison considers scientific documents from the period 2008–2017 covered by these data sources. Scopus is compared in a pairwise manner with each of the other data sources. We first analyze differences between the data sources in the coverage of documents, focusing for instance on differences over time, differences per document type, and differences per discipline. We then study differences in the completeness and accuracy of citation links. Based on our analysis, we discuss the strengths and weaknesses of the different data sources. We emphasize the importance of combining a comprehensive coverage of the scientific literature with a flexible set of filters for making selections of the literature."
2128125167,Self-citations at the meso and individual levels: effects of different calculation methods,"This paper focuses on the study of self-citations at the meso and micro (individual) levels, on the basis of an analysis of the production (1994-2004) of individual researchers working at the Spanish CSIC in the areas of Biology and Biomedicine and Material Sciences. Two different types of self-citations are described: author self-citations (citations received from the author him/herself) and co-author self-citations (citations received from the researchers' co-authors but without his/her participation). Self-citations do not play a decisive role in the high citation scores of documents either at the individual or at the meso level, which are mainly due to external citations. At micro-level, the percentage of self-citations does not change by professional rank or age, but differences in the relative weight of author and co-author self-citations have been found. The percentage of co-author self-citations tends to decrease with age and professional rank while the percentage of author self-citations shows the opposite trend. Suppressing author self-citations from citation counts to prevent overblown self-citation practices may result in a higher reduction of citation numbers of old scientists and, particularly, of those in the highest categories. Author and co-author self-citations provide valuable information on the scientific communication process, but external citations are the most relevant for evaluative purposes. As a final recommendation, studies considering self-citations at the individual level should make clear whether author or total self-citations are used as these can affect researchers differently."
1562168738,A Multilevel Modelling Approach to Investigating the Predictive Validity of Editorial Decisions: Do the Editors of a High Profile Journal Select Manuscripts that are Highly Cited After Publication?,"Summary Scientific journals must deal with the following questions concerning the predictive validity of editorial decisions. Is the best scientific work selected from submitted manuscripts? Does selection of the best manuscripts also mean selecting papers that after publication show top citation performance within their fields? Taking the journal Angewandte Chemie International Edition as an example, this study proposes a new methodology for investigating whether manuscripts that are most worthy of publication are in fact selected validly. First, the influence on citation of the accepted and rejected but then published elsewhere manuscripts was appraised on the basis of percentile impact classes scaled in a subfield of chemistry and, second, the association between the decisions on selection and the influence on citation of the manuscripts was determined by using a multilevel logistic regression for ordinal categories. This approach has many advantages over methodologies that were used in previous research studies on the predictive validity of editorial selection decisions."
4256597070,Who's Afraid of Peer Review?,A spoof paper concocted by Science reveals little or no scrutiny at many open-access journals.
2057809322,Gender differences in scientific productivity: a persisting phenomenon?,"There is substantial literature on research performance differences between male and female researchers, and its explanation. Using publication records of 852 social scientists, we show that performance differences indeed exist. However, our case study suggests that in the younger generation of researchers these have disappeared. If performance differences exist at all in our case, young female researchers outperform young male researchers. The trend in developed societies, that women increasingly outperform men in all levels of education, is also becoming effective in the science system."
2933830696,"Comprehensive Researcher Achievement Model (CRAM): a framework for measuring researcher achievement, impact and influence derived from a systematic literature review of metrics and models","Effective researcher assessment is key to decisions about funding allocations, promotion and tenure. We aimed to identify what is known about methods for assessing researcher achievements, leading to a new composite assessment model. We systematically reviewed the literature via the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols framework. All Web of Science databases (including Core Collection, MEDLINE and BIOSIS Citation Index) to the end of 2017. ELIGIBILITY CRITERIA: (1) English language, (2) published in the last 10 years (2007-2017), (3) full text was available and (4) the article discussed an approach to the assessment of an individual researcher's achievements. Articles were allocated among four pairs of reviewers for screening, with each pair randomly assigned 5% of their allocation to review concurrently against inclusion criteria. Inter-rater reliability was assessed using Cohen's Kappa (ĸ). The ĸ statistic showed agreement ranging from moderate to almost perfect (0.4848-0.9039). Following screening, selected articles underwent full-text review and bias was assessed. Four hundred and seventy-eight articles were included in the final review. Established approaches developed prior to our inclusion period (eg, citations and outputs, h-index and journal impact factor) remained dominant in the literature and in practice. New bibliometric methods and models emerged in the last 10 years including: measures based on PageRank algorithms or 'altmetric' data, methods to apply peer judgement and techniques to assign values to publication quantity and quality. Each assessment method tended to prioritise certain aspects of achievement over others. All metrics and models focus on an element or elements at the expense of others. A new composite design, the Comprehensive Researcher Achievement Model (CRAM), is presented, which supersedes past anachronistic models. The CRAM is modifiable to a range of applications."
2090841853,Prevalence of Honorary and Ghost Authorship in Cochrane Reviews,"To determine the prevalence of honorary and ghost authorship in Cochrane reviews, how authorship is assigned, and the ways in which authors and Cochrane editorial teams contribute.Using a Web-based, self-administered survey, corresponding authors for 577 reviews published in issues 1 and 2 from 1999 of The Cochrane Library were invited to report on the prevalence of honorary and ghost authors, contributions by authors listed in the byline and members of Cochrane editorial teams, and identification of methods of assigning authorship. Responses were received for 362 reviews (63% response rate), which contained 913 authors.One hundred forty-one reviews (39%) had evidence of honorary authors, 32 (9%) had evidence of ghost authors (most commonly a member of the Cochrane editorial team), and 9 (2%) had evidence of both honorary and ghost authors. The editorial teams contributed in a wide variety of ways to 301 reviews (83%). Authorship was decided by the group of authors (31%) or lead author (25%) in most reviews. Authorship order was assigned according to contribution in most reviews (76%). The 3 functions contributed to most by those listed in the byline were assessing the quality of included studies (83%), interpreting data (82%), and abstracting data from included studies (77%).A substantial proportion of reviews had evidence of honorary and ghost authorship. The Cochrane editorial teams contributed to most Cochrane reviews."
2551122076,The Global Burden of Journal Peer Review in the Biomedical Literature: Strong Imbalance in the Collective Enterprise,"The growth in scientific production may threaten the capacity for the scientific community to handle the ever-increasing demand for peer review of scientific publications. There is little evidence regarding the sustainability of the peer-review system and how the scientific community copes with the burden it poses. We used mathematical modeling to estimate the overall quantitative annual demand for peer review and the supply in biomedical research. The modeling was informed by empirical data from various sources in the biomedical domain, including all articles indexed at MEDLINE. We found that for 2015, across a range of scenarios, the supply exceeded by 15% to 249% the demand for reviewers and reviews. However, 20% of the researchers performed 69% to 94% of the reviews. Among researchers actually contributing to peer review, 70% dedicated 1% or less of their research work-time to peer review while 5% dedicated 13% or more of it. An estimated 63.4 million hours were devoted to peer review in 2015, among which 18.9 million hours were provided by the top 5% contributing reviewers. Our results support that the system is sustainable in terms of volume but emphasizes a considerable imbalance in the distribution of the peer-review effort across the scientific community. Finally, various individual interactions between authors, editors and reviewers may reduce to some extent the number of reviewers who are available to editors at any point."
1992861449,Diversity of references as an indicator of the interdisciplinarity of journals: Taking similarity between subject fields into account,"The objective of this article is to further the study of journal interdisciplinarity, or, more generally, knowledge integration at the level of individual articles. Interdisciplinarity is operationalized by the diversity of subject fields assigned to cited items in the article's reference list. Subject fields and subfields were obtained from the L euven‐ B udapest ( ECOOM ) subject‐classification scheme, while disciplinary diversity was measured taking variety, balance, and disparity into account. As diversity measure we use a H ill‐type true diversity in the sense of J ost and L einster‐ C obbold. The analysis is conducted in 3 steps. In the first part, the properties of this measure are discussed, and, on the basis of these properties it is shown that the measure has the potential to serve as an indicator of interdisciplinarity. In the second part the applicability of this indicator is shown using selected journals from several research fields ranging from mathematics to social sciences. Finally, the often‐heard argument, namely, that interdisciplinary research exhibits larger visibility and impact, is studied on the basis of these selected journals. Yet, as only 7 journals, representing a total of 15,757 articles, are studied, albeit chosen to cover a large range of interdisciplinarity, further research is still needed."
2109746829,Principles of scientific research team formation and evolution,"Research teams are the fundamental social unit of science, and yet there is currently no model that describes their basic property: size. In most fields, teams have grown significantly in recent decades. We show that this is partly due to the change in the character of team size distribution. We explain these changes with a comprehensive yet straightforward model of how teams of different sizes emerge and grow. This model accurately reproduces the evolution of empirical team size distribution over the period of 50 y. The modeling reveals that there are two modes of knowledge production. The first and more fundamental mode employs relatively small, ""core"" teams. Core teams form by a Poisson process and produce a Poisson distribution of team sizes in which larger teams are exceedingly rare. The second mode employs ""extended"" teams, which started as core teams, but subsequently accumulated new members proportional to the past productivity of their members. Given time, this mode gives rise to a power-law tail of large teams (10-1,000 members), which features in many fields today. Based on this model, we construct an analytical functional form that allows the contribution of different modes of authorship to be determined directly from the data and is applicable to any field. The model also offers a solid foundation for studying other social aspects of science, such as productivity and collaboration."
2952981920,Scopus's source normalized impact per paper (SNIP) versus a journal impact factor based on fractional counting of citations,"Abstract Impact factors (and similar measures such as the Scimago Journal Rankings) suffer from two problems: (a) citation behavior varies among fields of science and, therefore, leads to systematic differences, and (b) there are no statistics to inform us whether differences are significant. The recently introduced “source normalized impact per paper” indicator of Scopus tries to remedy the first of these two problems, but a number of normalization decisions are involved, which makes it impossible to test for significance. Using fractional counting of citations—based on the assumption that impact is proportionate to the number of references in the citing documents—citations can be contextualized at the paper level and aggregated impacts of sets can be tested for their significance. It can be shown that the weighted impact of Annals of Mathematics (0.247) is not so much lower than that of Molecular Cell (0.386) despite a five‐f old difference between their impact factors (2.793 and 13.156, respectively)."
2144692353,Understanding Bibliometric Parameters and Analysis,"Bibliometric parameters have become an important part of modern assessment of academic productivity. These parameters exist for the purpose of evaluating authors (publication count, citation count, h-index, m-quotient, hc-index, e-index, g-index, i-10 [i-n] index) and journals (impact factor, Eigenfactor, article influence score, SCImago journal rank, source-normalized impact per paper). Although in recent years there has been a proliferation of bibliometric parameters, the true meaning and appropriate use of these parameters is generally not well understood. Effective use of existing and emerging bibliometric tools can aid in assessment of academic productivity, including readiness for promotions and other awards. However, if not properly understood, the data can be misinterpreted and may be subject to manipulation. Familiarity with bibliometric parameters will aid in their effective implementation in the review of authors-whether individuals or groups-and journals, as well as their possible use in the promotions review process, maximizing the effectiveness of bibliometric analysis."
3172167978,A review on h-index and its alternative indices,"In recent years, several scientometrics and bibliometrics indicators were proposed to evaluate the scientific impact of individuals, institutions, colleges, universities and research teams. The h-index gives a breakthrough in the research community for assessing the scientific impact of an individual. It got a lot of attention due to its simplicity, and several other indicators were proposed to extend the properties of the h-index and to overcome its shortcomings. In this literature review, we have discussed the advantages and limitations of almost all scientometrics and bibliometrics indicators, which have been categorised into seven categories based on their properties: (1) complement of h-index, (2) based on total number of authors, (3) based on publication age, (4) combination of two indices, (5) based on excess citation count, (6) based on total publication count and (7) based on other variants. The primary objective of this article is to study all those indicators which have been proposed to evaluate the scientific impact of an individual researcher or a group of researchers."
1982126403,Global maps of science based on the new Web-of-Science categories,"In August 2011, Thomson Reuters launched version 5 of the Science and Social Science Citation Index in the Web of Science (WoS). Among other things, the 222 ISI Subject Categories (SCs) for these two databases in version 4 of WoS were renamed and extended to 225 WoS Categories (WCs). A new set of 151 Subject Areas was added, but at a higher level of aggregation. Perhaps confusingly, these Subject Areas are now abbreviated ""SC"" in the download, whereas ""WC"" is used for WoS Categories. Since we previously used the ISI SCs as the baseline for a global map in Pajek (Pajek is freely available at http://vlado.fmf.uni-lj.si/pub/networks/pajek/) (Rafols et al., Journal of the American Society for Information Science and Technology 61:1871-1887, 2010) and brought this facility online (at http://www.leydesdorff.net/overlaytoolkit), we recalibrated this map for the new WC categories using the Journal Citation Reports 2010. In the new installation, the base maps can also be made using VOSviewer (VOSviewer is freely available at http://www.VOSviewer.com/) (Van Eck and Waltman, Scientometrics 84:523-538, 2010)."
2102661446,Google Scholar revisited,"Purpose The purpose of this paper is to revisit Google Scholar. Design/methodology/approach This paper discusses the strengths and weaknesses of Google Scholar. Findings The Google Books project has given a massive and valuable boost to the already rich and diverse content of Google Scholar. The downside of the growth is that significant gaps remain for top ranking journals and serials, and the number of duplicate, triplicate and quadruplicate records for the same source documents (which Google Scholar cannot detect reliably) has increased. Originality/value This paper discusses the strengths and weaknesses of Google Scholar."
4237421641,Superstar Extinction*,"We estimate the magnitude of spillovers generated by 112 academic ""superstars"" who died prematurely and unexpectedly, thus providing an exogenous source of variation in the structure of their collaborators' coauthorship networks.Following the death of a superstar, we find that collaborators experience, on average, a lasting 5% to 8% decline in their quality-adjusted publication rates.By exploring interactions of the treatment effect with a variety of star, coauthor, and star/coauthor dyad characteristics, we seek to adjudicate between plausible mechanisms that might explain this finding.Taken together, our results suggest that spillovers are circumscribed in idea space, but less so in physical or social space.In particular, superstar extinction reveals the boundaries of the scientific field to which the star contributes-the ""invisible college.""""Greater is the merit of the person who facilitates the accomplishments of others than of the person who accomplishes himself."""
2800975350,Problems and challenges of predatory journals,"Abstract The companies publishing predatory journals are an emerging problem in the area of scientific literature as they only seek to drain money from authors without providing any customer service for the authors or their readership. These predatory journals try to attract new submissions by aggressive email advertising and high acceptance rates. But in turn, they do not provide proper peer review, and therefore, the scientific quality of submitted articles is questionable. This is important because more and more people, including patients, are reading such journals and rely on the information they provide. Consequently, predatory journals are a serious threat to the integrity of medical science, and it is crucial for scientists, physicians and even patients to be aware of this problem. In this review, we briefly summarize the history of the open access movement, as well as the rise of and roles played by predatory journals. In conclusion, young and inexperienced authors publishing in a predatory journal must be aware of the damage of their reputation, of inadequate peer review processes and that unprofitable journals might get closed and all published articles in that journal might be lost."
3125276829,Funding acknowledgements in scientific publications: A literature review,"Abstract The topic of acknowledgements has produced abundant research since the 1970s, though, as previous studies point out, the value of acknowledgements has not yet been demonstrated and further research is limited by lack of conceptualization. This study focuses on funding acknowledgements (FAs), considering that funding represents an important input in the scientific process. In this context, 183 scientific publications retrieved from Scopus from the 1970s until June 2020 were analyzed, with the aim of systematizing conceptually this body of research and contributing to a theory of acknowledgements. Results are summarized into the following main themes: the meaning of FAs; data sources for acknowledgements; the process of funding; association of funding with productivity, impact, and collaboration; and other aspects affected by funding. The literature reviewed shows that a theory of acknowledgements based on the reward triangle, as in previous studies, is unable to capture the extreme complexity of the scientific activity affecting and being affected by FAs. Funding bodies appear as clear and influential actors in the scientific communication system, making important decisions on the research that is supported, and influencing the type of knowledge produced. Funding agencies hold a responsibility regarding the data that they may collect on their programs, as well as the normalization policies they need to develop so that funded authors can reference with less ambiguity the financial source of their projects. Finally, the need to assess the impact of research funding beyond the scientific community that is, the societal impact, is also addressed."
2157485171,When are readership counts as useful as citation counts? Scopus versus Mendeley for LIS journals,"In theory, articles can attract readers on the social reference sharing site M endeley before they can attract citations, so M endeley altmetrics could provide early indications of article impact. This article investigates the influence of time on the number of M endeley readers of an article through a theoretical discussion and an investigation into the relationship between counts of readers of, and citations to, 4 general library and information science ( LIS ) journals. For this discipline, it takes about 7 years for articles to attract as many S copus citations as M endeley readers, and after this the S pearman correlation between readers and citers is stable at about 0.6 for all years. This suggests that M endeley readership counts may be useful impact indicators for both newer and older articles. The lack of dates for individual M endeley article readers and an unknown bias toward more recent articles mean that readership data should be normalized individually by year, however, before making any comparisons between articles published in different years."
3015626609,How much is too much? The difference between research influence and self-citation excess,"Abstract Citations can be an indicator of publication significance, utility, attention, visibility or short-term impact but analysts need to confirm whether a high citation count for an individual is a genuine reflection of influence or a consequence of extraordinary, even excessive, self-citation. It has recently been suggested there may be increasing misrepresentation of research performance by individuals who self-cite inordinately to achieve scores and win rewards. In this paper we consider self-referencing and self-citing, describe the typical shape of self-citation patterns for carefully curated publication sets authored by 3517 Highly Cited Researchers and quantify the variance in the distribution of self-citation rates within and between all 21 Essential Science Indicators’ fields. We describe both a generic level of median self-referencing rates, common to most fields, and a graphical, distribution-driven assessment of excessive self-citation that demarcates a threshold not dependent on statistical tests or percentiles (since for some fields all values are within a central ‘normal’ range). We describe this graphical procedure for identifying exceptional self-citation rates but emphasize the necessity for expert interpretation of the citation profiles of specific individuals, particularly in fields with atypical self-citation patterns."
4211244701,Analysis and Visualization of Citation Networks,"Citation analysis—the exploration of reference patterns in the scholarly and scientific literature—has long been applied in a number of social sciences to study research impact, knowledge flows, and k"
2052899782,Making Reviewers Visible,"Anonymity for peer reviewers remains the overwhelming norm within biomedical journals. While acknowledging that open review is not without challenges, this article presents 4 key arguments in its favor: (1) ethical superiority, (2) lack of important adverse effects, (3) feasibility in practice, and (4) potential to balance greater accountability for reviewers with credit for the work they do. Barriers to more widespread use of open review include conservatism within the research community and the fact that openness makes editors publicly responsible for their choice of reviewers and their interpretation of reviewers' comments. Forces for change include the growing use of preprint servers combined with open commentary. I look forward to a time when open commentary and review replace the current, flawed system of closed prepublication peer review and its false reassurances about the reliability of what is published."
2800788690,Governing Science,"Abstract This review explores contributions by science policy studies and the sociology of science to our understanding of the impact of governance on research content. Contributions are subsumed under two perspectives, namely an “impact of”—perspective that searches for effects of specific governance arrangements and an “impact on”—perspective that asks what factors contribute to the construction of research content and includes governance among them. Our review shows that little is known so far about the impact of governance on knowledge content. A research agenda does not necessarily need to include additional empirical phenomena but must address the macro-micro-macro link inherent to the question in its full complexity, and systematically exploit comparative approaches in order to establish causality. This requires interdisciplinary collaboration between science policy studies, the sociology of science, and bibliometrics, which all can contribute to the necessary analytical toolbox."
2041347735,Ring a Ring o’ Roses: Quality Journals and Gamesmanship in Management Studies*,"abstract A paper in one of the quality journals of Management Studies is much more important as a unit of measurement than as a contribution to knowledge. It measures academic performance and determines much academic funding. There is consequently some pressure to publish in quality journals. But quality journals are defined in terms that are themselves defined in terms of quality journals – a circularity that explains both the paper's title and the frustration of those who do not mix in these circles. We examine the gamesmanship of publishing in quality journals. Findings from a survey of heads of Management Studies departments in UK universities suggest that such gamesmanship is common. Cunning and calculation now support scholarship in Management Studies. Gamesmanship will remain common until the rewards for publishing attach to the content of papers, to what is published rather than where it is published. We propose a ‘Tinkerbell Solution’: without belief in the value of a paper in a quality journal, the game is no longer worth playing."
2142709701,Why do peer reviewers decline to review? A survey,"Peer reviewers are usually unpaid and their efforts not formally acknowledged. Some journals have difficulty finding appropriate reviewers able to complete timely reviews, resulting in publication delay.A survey of peer reviewers from five biomedical journals was conducted to determine why reviewers decline to review and their opinions on reviewer incentives. Items were scored on 5-point Likert scales, with low scores indicating low importance or low agreement.551/890 (62%) reviewers responded. Factors rated most highly in importance for the decision to accept to review a paper included contribution of the paper to subject area (mean 3.67 (standard deviation (SD) 86)), relevance of topic to own work (mean 3.46 (SD 0.99)) and opportunity to learn something new (mean 3.41 (SD 0.96)). The most highly rated factor important in the decision to decline to review was conflict with other workload (mean 4.06 (SD 1.31)). Most respondents agreed that financial incentives would not be effective when time constraints are prohibitive (mean 3.59 (SD 1.01)). However, reviewers agreed that non-financial incentives might encourage reviewers to accept requests to review: free subscription to journal content (mean 3.72 (SD 1.04)), annual acknowledgement on the journal's website (mean 3.64 (SD 0.90)), more feedback about the outcome of the submission (mean 3.62 (SD 0.88)) and quality of the review (mean 3.60 (SD 0.89), and appointment of reviewers to the journal's editorial board (mean 3.57 (SD 0.99)).Reviewers are more likely to accept to review a manuscript when it is relevant to their area of interest. Lack of time is the principal factor in the decision to decline. Reviewing should be formally recognised by academic institutions and journals should acknowledge reviewers' work."
2031821281,Plastic Surgery Classics: Characteristics of 50 Top-Cited Articles in Four Plastic Surgery Journals since 1946,"Citation of published articles by peers provides an indication of the relevance of the scientific work. Still, it is unknown what kinds of plastic surgery articles are cited most often. The authors set out to identify the characteristics of the 50 top-cited articles as published in four international, peer-reviewed, PubMed-indexed general plastic surgery journals.The 50 most-cited articles were identified in each of the following journals: Plastic and Reconstructive Surgery, the British Journal of Plastic Surgery, the Annals of Plastic Surgery, and the Scandinavian Journal of Plastic and Reconstructive Surgery and Hand Surgery. These 200 articles were ranked after their citation index, defined as the mean number of times they were cited per year during the first 16 years after publication. The top-50 articles thus ranked were analyzed for citation and journal distribution, geographic and institutional origin, surgical and anatomical subject, and level of evidence.Forty-one of the 50 top-cited articles (82 percent) were published in Plastic and Reconstructive Surgery and 35 articles (70 percent) originated from institutions within the United States. Most of the articles dealt with the reconstruction of acquired defects (45 percent) and with basic or experimental research (41 percent). Research that offered means for clinical improvement, rather than a high level of evidence or the results of multi-institutional collaboration, was most often cited.An article featuring a clinical or nonclinical innovation, observation, or discovery that leads to clinical improvement has the best potential to become a ""classic."""
2117456610,Scholarly Electronic Publishing Bibliography,"This bibliography presents selected articles, books, electronic documents, and other sources that are useful in understanding scholarly electronic publishing efforts on the Internet and other networks."
2950146322,Co‐occurrence matrices and their applications in information science: Extending ACA to the Web environment,"Abstract Co‐occurrence matrices, such as cocitation, coword, and colink matrices, have been used widely in the information sciences. However, confusion and controversy have hindered the proper statistical analysis of these data. The underlying problem, in our opinion, involved understanding the nature of various types of matrices. This article discusses the difference between a symmetrical cocitation matrix and an asymmetrical citation matrix as well as the appropriate statistical techniques that can be applied to each of these matrices, respectively. Similarity measures (such as the Pearson correlation coefficient or the cosine) should not be applied to the symmetrical cocitation matrix but can be applied to the asymmetrical citation matrix to derive the proximity matrix. The argument is illustrated with examples. The study then extends the application of co‐occurrence matrices to the Web environment, in which the nature of the available data and thus data collection methods are different from those of traditional databases such as the Science Citation Index . A set of data collected with the Google Scholar search engine is analyzed by using both the traditional methods of multivariate analysis and the new visualization software Pajek, which is based on social network analysis and graph theory."
2161769520,Open Access Publishing: A Literature Review,"Within the context of the Centre for Copyright and New Business Models in the Creative Economy (CREATe) research scope, this literature review investigates the current trends, advantages, disadvantages, problems and solutions, opportunities and barriers in Open Access Publishing (OAP), and in particular Open Access (OA) academic publishing. This study is intended to scope and evaluate current theory and practice concerning models for OAP and engage with intellectual, legal and economic perspectives on OAP. It is also aimed at mapping the field of academic publishing in the UK and abroad, drawing specifically upon the experiences of CREATe industry partners as well as other initiatives such as SSRN, open source software, and Creative Commons. As a final critical goal, this scoping study will identify any meaningful gaps in the relevant literature with a view to developing further research questions. The results of this scoping exercise will then be presented to relevant industry and academic partners at a workshop intended to assist in further developing the critical research questions pertinent to OAP."
2527829392,"The academic, economic and societal impacts of Open Access: an evidence-based review","Ongoing debates surrounding Open Access to the scholarly literature are multifaceted and complicated by disparate and often polarised viewpoints from engaged stakeholders. At the current stage, Open Access has become such a global issue that it is critical for all involved in scholarly publishing, including policymakers, publishers, research funders, governments, learned societies, librarians, and academic communities, to be well-informed on the history, benefits, and pitfalls of Open Access. In spite of this, there is a general lack of consensus regarding the potential pros and cons of Open Access at multiple levels. This review aims to be a resource for current knowledge on the impacts of Open Access by synthesizing important research in three major areas: academic, economic and societal. While there is clearly much scope for additional research, several key trends are identified, including a broad citation advantage for researchers who publish openly, as well as additional benefits to the non-academic dissemination of their work. The economic impact of Open Access is less well-understood, although it is clear that access to the research literature is key for innovative enterprises, and a range of governmental and non-governmental services. Furthermore, Open Access has the potential to save both publishers and research funders considerable amounts of financial resources, and can provide some economic benefits to traditionally subscription-based journals. The societal impact of Open Access is strong, in particular for advancing citizen science initiatives, and leveling the playing field for researchers in developing countries. Open Access supersedes all potential alternative modes of access to the scholarly literature through enabling unrestricted re-use, and long-term stability independent of financial constraints of traditional publishers that impede knowledge sharing. However, Open Access has the potential to become unsustainable for research communities if high-cost options are allowed to continue to prevail in a widely unregulated scholarly publishing market. Open Access remains only one of the multiple challenges that the scholarly publishing system is currently facing. Yet, it provides one foundation for increasing engagement with researchers regarding ethical standards of publishing and the broader implications of 'Open Research'."
2094898550,Nanoscience and technology publications and patents: a review of social science studies and search strategies,"We undertake a comprehensive review of more than 120 social science studies on nanotechnology, 90% of which are based on the analyses of the nanotechnology publications and patents. We discussed four intellectual debates formed by these studies, namely whether nanotechnology is an interdisciplinary field, whether nanoscience and nanotechnology are closely interlinked, whether nanotechnology development is path dependent and who is winning the global nanorace. We also conduct a comparative analysis of bibliometric search strategies used in the literature to harvest the publications and patents, including lexical queries, evolutionary lexical queries, citation analysis, and the use of core journal sets to identify nanotechnology articles. Because most of the compared strategies, except the one using 10 core journals in the field, share a core set of keywords and thus harvest a common batch of publications, they produce very similar ranking tables of the top subject areas and journals and the most prolific countries and institutions. Moreover, the core journal strategy does not provide a robust delineation of an emerging field such as nanotechnology due to the fact that nanotechnology related articles are published in a wide range of journals. Also, the different criteria for selecting the core journals will affect the analytical results dramatically."
2142098429,Collaboration Paradox,"We examine the ways in which the research process differs in developed and developing areas by focusing on two questions. First, is collaboration associated with productivity? Second, is access to the Internet (specifically use of email) associated with reduced problems of collaboration? Recent analyses by Lee & Bozeman (2005) and Walsh & Maloney (2003) suggest affirmative answers to these questions for US scientists. Based on a comparative analysis of scientists in Ghana, Kenya, and the State of Kerala in south-western India ( N 918), we find that: (1) collaboration is not associated with any general increment in productivity; and (2) while access to email does attenuate research problems, such difficulties are structured more by national and regional context than by the collaborative process itself. The interpretation of these results suggests a paradox that raises issues for future studies: those conditions that unsettle the relationship between collaboration and productivity in developing areas may undermine the collaborative benefits of new information and communication technologies."
3014920224,The use of bibliometric analysis in research performance assessment and monitoring of interdisciplinary scientific developments,"This paper1 presents an overview of advanced bibliometric methods for (1) objective and transparent assessment of strengths and weaknesses in research performance, and (2) monitoring interdisciplinary scientific developments.In the first application, we focus on the detailed analysis of research performance in an international comparative perspective.We demonstrate that advanced bibliometric methods are, particularly at the level of research groups, university departments and institutes, an indispensable element next to peer review in research evaluation procedures.We address specific problems for the social sciences.In the second application, monitoring of scientific (basic and applied) developments, recent advances in bibliometric mapping techniques are promising.They are unique instruments to discover patterns in the structure of scientific fields, to identify processes of knowledge dissemination, and to visualize the dynamics of scientific developments.We discuss briefly their potential for unraveling interdisciplinary developments and interfaces between science and technology."
2109976502,Effect of Blinded Peer Review on Abstract Acceptance,"Peer review should evaluate the merit and quality of abstracts but may be biased by geographic location or institutional prestige. The effectiveness of blinded peer review at reducing bias is unknown. To evaluate the effect of blinded review on the association between abstract characteristics and likelihood of abstract acceptance at a national research meeting. All abstracts submitted to the American Heart Association's annual Scientific Sessions research meeting from 2000-2004. Abstract review included the author's name and institution (open review) from 2000-2001, and this information was concealed (blinded review) from 2002-2004. Abstracts were categorized by country, primary language, institution prestige, author sex, and government and industry status. Likelihood of abstract acceptance during open and blinded review, by abstract characteristics. The mean number of abstracts submitted each year for evaluation was 13,455 and 28.5% were accepted. During open review, 40.8% of US and 22.6% of non-US abstracts were accepted (relative risk [RR], 1.81; 95% confidence interval [CI], 1.75-1.88), whereas during blinded review, 33.4% of US and 23.7% of non-US abstracts were accepted (RR, 1.41; 95% CI, 1.37-1.45; P<.001 for comparison between peer review periods). Among non-US abstracts, during open review, 31.1% from English- speaking countries and 20.9% from non-English-speaking countries were accepted (RR, 1.49; 95% CI, 1.39-1.59), whereas during blinded review, 28.8% and 22.8% of abstracts were accepted, respectively (RR, 1.26; 95% CI, 1.19-1.34; P<.001). Among abstracts from US academic institutions, during open review, 51.3% from highly prestigious and 32.6% from nonprestigious institutions were accepted (RR, 1.57; 95% CI, 1.48-1.67), whereas during blinded review, 38.8% and 29.0% of abstracts were accepted, respectively (RR, 1.34; 95% CI, 1.26-1.41; P<.001). This study provides evidence of bias in the open review of abstracts, favoring authors from the United States, English-speaking countries outside the United States, and prestigious academic institutions. Moreover, blinded review at least partially reduced reviewer bias."
2055458526,"Africa’s contribution to the worldwide research literature: New analytical perspectives, trends, and performance indicators","This paper examines general characteristics of African science from a quantitative 'scientometric' perspective. More specifically, that of research outputs of Africa-based authors published in the scientific literature during the years 1980–2004, either within the international journals representing 'mainstream' science, or within national and regional journals reflecting 'indigenous science'. As for the international journals, the findings derived from Thomson Scientific's Citation Indexes show that while Africa's share in worldwide science has steadily declined, the share of international co-publications has increased very significantly, whereas low levels of international citation impact persist. A case study of South African journals reveals the existence of several journals that are not processed for these international databases but nonetheless show a distinctive citation impact on international research communities."
2063743914,A Reliability-Generalization Study of Journal Peer Reviews: A Multilevel Meta-Analysis of Inter-Rater Reliability and Its Determinants,"Background This paper presents the first meta-analysis for the inter-rater reliability (IRR) of journal peer reviews. IRR is defined as the extent to which two or more independent reviews of the same scientific document agree. Methodology/Principal Findings Altogether, 70 reliability coefficients (Cohen's Kappa, intra-class correlation [ICC], and Pearson product-moment correlation [r]) from 48 studies were taken into account in the meta-analysis. The studies were based on a total of 19,443 manuscripts; on average, each study had a sample size of 311 manuscripts (minimum: 28, maximum: 1983). The results of the meta-analysis confirmed the findings of the narrative literature reviews published to date: The level of IRR (mean ICC/r2 = .34, mean Cohen's Kappa = .17) was low. To explain the study-to-study variation of the IRR coefficients, meta-regression analyses were calculated using seven covariates. Two covariates that emerged in the meta-regression analyses as statistically significant to gain an approximate homogeneity of the intra-class correlations indicated that, firstly, the more manuscripts that a study is based on, the smaller the reported IRR coefficients are. Secondly, if the information of the rating system for reviewers was reported in a study, then this was associated with a smaller IRR coefficient than if the information was not conveyed. Conclusions/Significance Studies that report a high level of IRR are to be considered less credible than those with a low level of IRR. According to our meta-analysis the IRR of peer assessments is quite limited and needs improvement (e.g., reader system)."
2058954808,The hybrid model for open access publication of scholarly articles: A failed experiment?,"Since 2004, mainstream scholarly publishers have been offering authors publishing in their subscription journals the option to free their individual articles from access barriers against a payment (hybrid OA ). This has been marketed as a possible gradual transition path between subscription and open access to the scholarly journal literature, and the publishers have pledged to decrease their subscription prices in proportion to the uptake of the hybrid option. The number of hybrid journals has doubled in the past couple of years and is now over 4,300; the number of such articles was around 12,000 in 2011. On average only 1–2% of eligible authors utilize the OA option, due mainly to the generally high price level of typically 3,000 USD . There are, however, a few publishers and individual journals with a much higher uptake. This article takes a closer look at the development of hybrid OA and discusses, from an author‐centric viewpoint, the possible reasons for the lack of success of this business model."
2049245574,The Effects of Aging on Researchers' Publication and Citation Patterns,"The average age at which U.S. researchers get their first grant from NIH has increased from 34.3 in 1970, to 41.7 in 2004. These data raise the crucial question of the effects of aging on the scientific creativity and productivity of researchers. Those who worry about the aging of scientists usually believe that the younger they are the more creative and productive they will be. Using a large population of 13,680 university professors in Quebec, we show that, while scientific productivity rises sharply between 28 and 40, it increases at a slower pace between 41 and 50 and stabilizes afterward until retirement for the most active researchers. The average scientific impact per paper decreases linearly until 50-55 years old, but the average number of papers in highly cited journals and among highly cited papers rises continuously until retirement. Our results clearly show for the first time the natural history of the scientific productivity of scientists over their entire career and bring to light the fact that researchers over 55 still contribute significantly to the scientific community by producing high impact papers."
2150499457,Authors as citers over time,"This study explores the tendency of authors to recite themselves and others in multiple works over time, using the insights gained to build citation theory. The set of all authors whom an author cites is defined as that author's citation identity. The study explains how to retrieve citation identities from the Institute for Scientific Information's files on Dialog and how to deal with idiosyncrasies of these files. As the author's oeuvre grows, the identity takes the form of a core-and-scatter distribution that may be divided into authors cited only once (unicitations) and authors cited at least twice (recitations). The latter group, especially those recited most frequently, are interpretable as symbols of a citer's main substantive concerns. As illustrated by the top recitees of eight information scientists, identities are intelligible, individualized, and wide-ranging. They are ego-centered without being egotistical. They are often affected by social ties between citers and citees, but the universal motivator seems to be the perceived relevance of the citees' works. Citing styles in identities differ: “scientific-paper style” authors recite heavily, adding to core; “bibliographic-essay style” authors are heavy on unicitations, adding to scatter; “literature-review style” authors do both at once. Identities distill aspects of citers' intellectual lives, such as orienting figures, interdisciplinary interests, bidisciplinary careers, and conduct in controversies. They can also be related to past schemes for classifying citations in categories such as positive–negative and perfunctory–organic; indeed, one author's frequent recitation of another, whether positive or negative, may be the readiest indicator of an organic relation between them. The shape of the core-and-scatter distribution of names in identities can be explained by the principle of least effort. Citers economize on effort by frequently reciting only a relatively small core of names in their identities. They also economize by frequent use of perfunctory citations, which require relatively little context, and infrequent use of negative citations, which require contexts more laborious to set."
3100614207,On the calculation of percentile‐based bibliometric indicators,"A percentile‐based bibliometric indicator is an indicator that values publications based on their position within the citation distribution of their field. The most straightforward percentile‐based indicator is the proportion of frequently cited publications, for instance, the proportion of publications that belong to the top 10% most frequently cited of their field. Recently, more complex percentile‐based indicators have been proposed. A difficulty in the calculation of percentile‐based indicators is caused by the discrete nature of citation distributions combined with the presence of many publications with the same number of citations. We introduce an approach to calculating percentile‐based indicators that deals with this difficulty in a more satisfactory way than earlier approaches suggested in the literature. We show in a formal mathematical framework that our approach leads to indicators that do not suffer from biases in favor of or against particular fields of science."
2002728047,"Testing the Calculation of a Realistic h -index in Google Scholar, Scopus, and Web of Science for F. W. Lancaster","This paper focuses on the practical limitations in the content and software of the databases that are used to calculate the h -index for assessing the publishing productivity and impact of researchers. To celebrate F. W. Lancaster’s biological age of seventy-five, and “scientific age” of forty-five, this paper discusses the related features of Google Scholar, Scopus, and Web of Science (WoS), and demonstrates in the latter how a much more realistic and fair h -index can be computed for F. W. Lancaster than the one produced automatically. Browsing and searching the cited reference index of the 1945–2007 edition of WoS, which in my estimate has over a hundred million “orphan references” that have no counterpart master records to be attached to, and “stray references” that cite papers which do have master records but cannot be identified by the matching algorithm because of errors of omission and commission in the references of the citing works, can bring up hundreds of additional cited references given to works of an accomplished author but are ignored in the automatic process of calculating the h -index. The partially manual process doubled the h -index value for F. W. Lancaster from 13 to 26, which is a much more realistic value for an information scientist and professor of his stature."
4240507605,The effect of “open access” on citation impact: An analysis of ArXiv's condensed matter section,"Abstract This article statistically analyzes how the citation impact of articles deposited in the Condensed Matter section of the preprint server ArXiv (hosted by Cornell University), and subsequently published in a scientific journal, compares to that of articles in the same journal that were not deposited in the archive. Its principal aim is to further illustrate and roughly estimate the effect of two factors, “early view” and “quality bias,” on differences in citation impact between these two sets of papers, using citation data from Thomson Scientific's Web of Science . It presents estimates for a number of journals in the field of condensed matter physics. To discriminate between an “open access” effect and an early view effect, longitudinal citation data were analyzed covering a time period as long as 7 years. Quality bias was measured by calculating ArXiv citation impact differentials at the level of individual authors publishing in a journal, taking into account coauthorship. The analysis provided evidence of a strong quality bias and early view effect. Correcting for these effects, there is in a sample of six condensed matter physics journals studied in detail no sign of a general “open access advantage” of papers deposited in ArXiv. The study does provide evidence that ArXiv accelerates citation due to the fact that ArXiv makes papers available earlier rather than makes them freely available."
2035104656,Peer Review and the Social Construction of Knowledge in the Management Discipline,"Prior research on the peer-review process has almost exclusively focused on its surface features--its impartiality, validity, and reliability. What has received relatively less attention is the influence of the social component that shapes the content of the discipline's published record and, in turn, determines its scientific progress. As the product of social processes, all knowledge-claims are socially constituted rather than the products of an absolute truth. Taking a sociology-of-knowledge perspective, I argue that the social processes underlying the peer-review process warrant closer scrutiny. In doing so, I contend that there must be a balancing of the inevitable author-editor-referee tensions operating throughout the editorial process so as to ensure that a clear authorial voice is preserved. I offer suggestions for assuring the integrity of the scientific enterprise, while respecting the prerogatives and ethics of authorship."
2075054625,The Possible Role of Resource Requirements and Academic Career-Choice Risk on Gender Differences in Publication Rate and Impact,"Many studies demonstrate that there is still a significant gender bias, especially at higher career levels, in many areas including science, technology, engineering, and mathematics (STEM). We investigated field-dependent, gender-specific effects of the selective pressures individuals experience as they pursue a career in academia within seven STEM disciplines. We built a unique database that comprises 437,787 publications authored by 4,292 faculty members at top United States research universities. Our analyses reveal that gender differences in publication rate and impact are discipline-specific. Our results also support two hypotheses. First, the widely-reported lower publication rates of female faculty are correlated with the amount of research resources typically needed in the discipline considered, and thus may be explained by the lower level of institutional support historically received by females. Second, in disciplines where pursuing an academic position incurs greater career risk, female faculty tend to have a greater fraction of higher impact publications than males. Our findings have significant, field-specific, policy implications for achieving diversity at the faculty level within the STEM disciplines."
2111836958,Predicting scientific success based on coauthorship networks,"We address the question to what extent the success of scientific articles is due to social influence. Analyzing a data set of over 100,000 publications from the field of Computer Science, we study how centrality in the coauthorship network differs between authors who have highly cited papers and those who do not. We further show that a Machine Learning classifier, based only on coauthorship network centrality metrics measured at the time of publication, is able to predict with high precision whether an article will be highly cited five years after publication. By this we provide quantitative insight into the social dimension of scientific publishing – challenging the perception of citations as an objective, socially unbiased measure of scientific success."
2888173901,The state of the art in peer review,"Scholarly communication is in a perpetual state of disruption. Within this, peer review of research articles remains an essential part of the formal publication process, distinguishing it from virtually all other modes of communication. In the last several years, there has been an explosive wave of innovation in peer review research, platforms, discussions, tools and services. This is largely coupled with the ongoing and parallel evolution of scholarly communication as it adapts to rapidly changing environments, within what is widely considered as the 'open research' or 'open science' movement. Here, we summarise the current ebb and flow around changes to peer review and consider its role in a modern digital research and communications infrastructure and suggest why uptake of new models of peer review appears to have been so low compared to what is often viewed as the 'traditional' method of peer review. Finally, we offer some insight into the potential futures of scholarly peer review and consider what impacts this might have on the broader scholarly research ecosystem. In particular, we focus on the key traits of certification and reputation, moderation and quality control and engagement incentives, and discuss how these interact with socio-technical aspects of peer review and academic culture."
1696956904,Multidimensional assessment of scholarly research impact,"This article introduces the Multidimensional Research Assessment Matrix of scientific output. Its base notion holds that the choice of metrics to be applied in a research assessment process depends on the unit of assessment, the research dimension to be assessed, and the purposes and policy context of the assessment. An indicator may by highly useful within one assessment process, but less so in another. For instance, publication counts are useful tools to help discriminate between those staff members who are research active, and those who are not, but are of little value if active scientists are to be compared with one another according to their research performance. This paper gives a systematic account of the potential usefulness and limitations of a set of 10 important metrics, including altmetrics, applied at the level of individual articles, individual researchers, research groups, and institutions. It presents a typology of research impact dimensions and indicates which metrics are the most appropriate to measure each dimension. It introduces the concept of a “meta‐analysis” of the units under assessment in which metrics are not used as tools to evaluate individual units, but to reach policy inferences regarding the objectives and general setup of an assessment process."
2093771631,The Misused Impact Factor,"Research papers from all over the world are published in thousands of Science journals every year. The quality of these papers clearly has to be evaluated, not only to determine their accuracy and contribution to fields of research, but also to help make informed decisions about rewarding scientists with funding and appointments to research positions. One measure often used to determine the quality of a paper is the so-called impact factor of the journal in which it was published. This citation-based metric is meant to rank scientific journals, but there have been numerous criticisms over the years of its use as a measure of the quality of individual research papers. Still, this misuse persists. Why?"
2145677467,Authorship Criteria and Disclosure of Contributions,"2][3] In response to the 1997 proposal to acknowledge research contributions in journal articles 1 as a way to limit irresponsible authorship, contribution"
2950748763,Transparency in authors’ contributions and responsibilities to promote integrity in scientific publication,"In keeping with the growing movement in scientific publishing toward transparency in data and methods, we propose changes to journal authorship policies and procedures to provide insight into which author is responsible for which contributions, better assurance that the list is complete, and clearly articulated standards to justify earning authorship credit. To accomplish these goals, we recommend that journals adopt common and transparent standards for authorship, outline responsibilities for corresponding authors, adopt the Contributor Roles Taxonomy (CRediT) ( docs.casrai.org/CRediT ) methodology for attributing contributions, include this information in article metadata, and require authors to use the ORCID persistent digital identifier ( https://orcid.org ). Additionally, we recommend that universities and research institutions articulate expectations about author roles and responsibilities to provide a point of common understanding for discussion of authorship across research teams. Furthermore, we propose that funding agencies adopt the ORCID identifier and accept the CRediT taxonomy. We encourage scientific societies to further authorship transparency by signing on to these recommendations and promoting them through their meetings and publications programs."
4244276741,Authors as citers over time,"This study explores the tendency of authors to recite themselves and others in multiple works over time, using the insights gained to build citation theory. The set of all authors whom an author cites is defined as that author's citation identity. The study explains how to retrieve citation identities from the Institute for Scientific Information's files on Dialog and how to deal with idiosyncrasies of these files. As the author's oeuvre grows, the identity takes the form of a core-and-scatter distribution that may be divided into authors cited only once (unicitations) and authors cited at least twice (recitations). The latter group, especially those recited most frequently, are interpretable as symbols of a citer's main substantive concerns. As illustrated by the top recitees of eight information scientists, identities are intelligible, individualized, and wide-ranging. They are ego-centered without being egotistical. They are often affected by social ties between citers and citees, but the universal motivator seems to be the perceived relevance of the citees' works. Citing styles in identities differ: ""scientific-paper style"" authors recite heavily, adding to core; ""bibliographic-essay style"" authors are heavy on unicitations, adding to scatter; ""literature-review style"" authors do both at once. Identities distill aspects of citers' intellectual lives, such as orienting figures, interdisciplinary interests, bidisciplinary careers, and conduct in controversies. They can also be related to past schemes for classifying citations in categories such as positive–negative and perfunctory–organic; indeed, one author's frequent recitation of another, whether positive or negative, may be the readiest indicator of an organic relation between them. The shape of the core-and-scatter distribution of names in identities can be explained by the principle of least effort. Citers economize on effort by frequently reciting only a relatively small core of names in their identities. They also economize by frequent use of perfunctory citations, which require relatively little context, and infrequent use of negative citations, which require contexts more laborious to set."
2785823074,Sci-Hub provides access to nearly all scholarly literature,"The website Sci-Hub enables users to download PDF versions of scholarly articles, including many articles that are paywalled at their journal’s site. Sci-Hub has grown rapidly since its creation in 2011, but the extent of its coverage has been unclear. Here we report that, as of March 2017, Sci-Hub’s database contains 68.9% of the 81.6 million scholarly articles registered with Crossref and 85.1% of articles published in toll access journals. We find that coverage varies by discipline and publisher, and that Sci-Hub preferentially covers popular, paywalled content. For toll access articles, we find that Sci-Hub provides greater coverage than the University of Pennsylvania, a major research university in the United States. Green open access to toll access articles via licit services, on the other hand, remains quite limited. Our interactive browser at https://greenelab.github.io/scihub allows users to explore these findings in more detail. For the first time, nearly all scholarly literature is available gratis to anyone with an Internet connection, suggesting the toll access business model may become unsustainable."
2005234471,Co-authorship networks: a review of the literature,"Purpose – The purpose of this paper is to attempt to provide a review of the growing literature on co-authorship networks and the research gaps that may be investigated for future studies in this field. Design/methodology/approach – The existing literature on co-authorship networks was identified, evaluated and interpreted. Narrative review style was followed. Findings – Co-authorship, a proxy of research collaboration, is a key mechanism that links different sets of talent to produce a research output. Co-authorship could also be seen from the perspective of social networks. An in-depth analysis of such knowledge networks provides an opportunity to investigate its structure. Patterns of these relationships could reveal, for example, the mechanism that shapes our scientific community. The study provides a review of the expanding literature on co-authorship networks. Originality/value – This is one of the first comprehensive reviews of network-based studies on co-authorship. The field is fast evolving, opening new gaps for potential research. The study identifies some of these gaps."
2026950120,Quantifying Coauthor Contributions,"The model of descriptive, diagnostic, predictive, and prescriptive analytics on 100 top-cited articles of nasopharyngeal carcinoma from 2013 to 2022: Bibliometric analysis,"
2070593969,Web of Science and Scopus: a journal title overlap study,"Purpose The purpose of this research is to provide the scientific community with some quantitative data of relevance to the evaluation of two major citation databases. In addition, various aspects of the methodology of database coverage comparisons are discussed. Design/methodology/approach Calculations of the overlaps between the journal lists of Web of Science and Scopus and some other major scientific databases are presented. Findings The results provide some measures of the overall title coverage as well as the amount of unique material in the sources studied. Research limitations/implications The journal title overlap calculations are based on journal lists provided by the database producers rather than searches in the databases themselves. Any inaccuracies in the lists may be reflected in the results. Also, the lists do not provide any information about the depth and consistency of the coverage. The nature of possible error sources is discussed. Originality/value The methodology chosen allows comparatively quick comparisons between the contents of databases. This makes it suitable for analysis of trends in database coverage."
2039278910,Escape from the impact factor,"ESEP Ethics in Science and Environmental Politics Contact the journal RSS Mailing List Subscribe to our mailing list via Mailchimp HomeLatest VolumeAbout the JournalEditorsTheme Sections ESEP 8:5-7 (2008) - DOI: https://doi.org/10.3354/esep00078 Escape from the impact factor Philip Campbell* Nature, 4 Crinan Street, London N1 9XW, UK *Email: nature@nature.com ABSTRACT: As Editor-in-Chief of the journal Nature, I am concerned by the tendency within academic administrations to focus on a journal's impact factor when judging the worth of scientific contributions by researchers, affecting promotions, recruitment and, in some countries, financial bonuses for each paper. Our own internal research demonstrates how a high journal impact factor can be the skewed result of many citations of a few papers rather than the average level of the majority, reducing its value as an objective measure of an individual paper. Proposed alternative indices have their own drawbacks. Many researchers say that their important work has been published in low-impact journals. Focusing on the citations of individual papers is a more reliable indicator of an individual's impact. A positive development is the increasing ability to track the contributions of individuals by means of author-contribution statements and perhaps, in the future, citability of components of papers rather than the whole. There are attempts to escape the hierarchy of high-impact-factor journals by means of undifferentiated databases of peer-reviewed papers such as PLoS One. It remains to be seen whether that model will help outstanding work to rise to due recognition regardless of editorial selectivity. Although the current system may be effective at measuring merit on national and institutional scales, the most effective and fair analysis of a person's contribution derives from a direct assessment of individual papers, regardless of where they were published. KEY WORDS: Bibliometrics · Impact factor · Academic merit Full text in pdf format PreviousNextCite this article as: Campbell P (2008) Escape from the impact factor. Ethics Sci Environ Polit 8:5-7. https://doi.org/10.3354/esep00078 Export citation RSS - Facebook - Tweet - linkedIn Cited by Published in ESEP Vol. 8, No. 1. Online publication date: June 03, 2008 Print ISSN: 1863-5415; Online ISSN: ISSN Copyright © 2008 Inter-Research."
2949130492,PageRank for ranking authors in co‐citation networks,"Abstract This paper studies how varied damping factors in the PageRank algorithm influence the ranking of authors and proposes weighted PageRank algorithms. We selected the 108 most highly cited authors in the information retrieval (IR) area from the 1970s to 2008 to form the author co‐citation network. We calculated the ranks of these 108 authors based on PageRank with the damping factor ranging from 0.05 to 0.95. In order to test the relationship between different measures, we compared PageRank and weighted PageRank results with the citation ranking, h‐index, and centrality measures. We found that in our author co‐citation network, citation rank is highly correlated with PageRank with different damping factors and also with different weighted PageRank algorithms; citation rank and PageRank are not significantly correlated with centrality measures; and h‐index rank does not significantly correlate with centrality measures but does significantly correlate with other measures. The key factors that have impact on the PageRank of authors in the author co‐citation network are being co‐cited with important authors."
2953193560,The impact factor's Matthew Effect: A natural experiment in bibliometrics,"Abstract Since the publication of Robert K. Merton's theory of cumulative advantage in science (Matthew Effect), several empirical studies have tried to measure its presence at the level of papers, individual researchers, institutions, or countries. However, these studies seldom control for the intrinsic “quality” of papers or of researchers—“better” (however defined) papers or researchers could receive higher citation rates because they are indeed of better quality. Using an original method for controlling the intrinsic value of papers—identical duplicate papers published in different journals with different impact factors—this paper shows that the journal in which papers are published have a strong influence on their citation rates, as duplicate papers published in high‐impact journals obtain, on average, twice as many citations as their identical counterparts published in journals with lower impact factors. The intrinsic value of a paper is thus not the only reason a given paper gets cited or not, there is a specific Matthew Effect attached to journals and this gives to papers published there an added value over and above their intrinsic quality."
4237854446,Characteristics of research output in social sciences and humanities: From a research evaluation perspective,"Abstract The goal of research evaluation is to reveal the achievement and progress of research. Research output offers a basis for empirical evaluation. A fair and just research evaluation should take into account the diversity of research output across disciplines and include all major forms of research publications. This article reviews the literature on the nature of research output in social sciences and humanities in terms of the characteristics of research publications, and then discusses the implications for the research evaluation of social sciences and humanities researchers."
2032402175,The cognitive structure of Library and Information Science: Analysis of article title words,"Abstract This study comprises a suite of analyses of words in article titles in order to reveal the cognitive structure of Library and Information Science (LIS). The use of title words to elucidate the cognitive structure of LIS has been relatively neglected. The present study addresses this gap by performing (a) co‐word analysis and hierarchical clustering, (b) multidimensional scaling, and (c) determination of trends in usage of terms. The study is based on 10,344 articles published between 1988 and 2007 in 16 LIS journals. Methodologically, novel aspects of this study are: (a) its large scale, (b) removal of non‐specific title words based on the “word concentration” measure (c) identification of the most frequent terms that include both single words and phrases, and (d) presentation of the relative frequencies of terms using “heatmaps”. Conceptually, our analysis reveals that LIS consists of three main branches: the traditionally recognized library‐related and information‐related branches, plus an equally distinct bibliometrics/scientometrics branch. The three branches focus on: libraries, information, and science, respectively. In addition, our study identifies substructures within each branch. We also tentatively identify “information seeking behavior” as a branch that is establishing itself separate from the three main branches. Furthermore, we find that cognitive concepts in LIS evolve continuously, with no stasis since 1992. The most rapid development occurred between 1998 and 2001, influenced by the increased focus on the Internet. The change in the cognitive landscape is found to be driven by the emergence of new information technologies, and the retirement of old ones."
2139696124,Life and times of the impact factor: retrospective analysis of trends for seven medical journals (1994-2005) and their Editors’ views,"(1) To analyse trends in the journal impact factor (IF) of seven general medical journals (Ann Intern Med, BMJ, CMAJ, JAMA, Lancet, Med J Aust and N Engl J Med) over 12 years; and (2) to ascertain the views of these journals' past and present Editors on factors that had affected their journals' IFs during their tenure, including direct editorial policies.Retrospective analysis of IF data from ISI Web of Knowledge Journal Citation Reports-Science Edition, 1994 to 2005, and interviews with Editors-in-Chief.Medical journal publishing.Ten Editors-in-Chief of the journals, except Med J Aust, who served between 1999 and 2004. MAIN OUTCOME MEASURES IFs and component numerator and denominator data for the seven general medical journals (1994 to 2005) were collected. IFs are calculated using the formula: (Citations in year z to articles published in years x and y)/(Number of citable articles published in years x and y), where z is the current year and x and y are the previous two years. Editors' views on factors that had affected their journals' IFs were also obtained.IFs generally rose over the 12-year period, with the N Engl J Med having the highest IF throughout. However, percentage rises in IF relative to the baseline year of 1994 were greatest for CMAJ (about 500%) and JAMA (260%). Numerators for most journals tended to rise over this period, while denominators tended to be stable or to fall, although not always in a linear fashion. Nine of ten eligible editors were interviewed. Possible reasons given for rises in citation counts included: active recruitment of high-impact articles by courting researchers; offering authors better services; boosting the journal's media profile; more careful article selection; and increases in article citations. Most felt that going online had not affected citations. Most had no deliberate policy to publish fewer articles (lowering the IF denominator), which was sometimes the unintended result of other editorial policies. The two Editors who deliberately published fewer articles did so as they realized IFs were important to authors. Concerns about the accuracy of ISI counting for the IF denominator prompted some to routinely check their IF data with ISI. All Editors had mixed feelings about using IFs to evaluate journals and academics, and mentioned the tension between aiming to improve IFs and 'keeping their constituents [clinicians] happy.'IFs of the journals studied rose in the 12-year period due to rising numerators and/or falling denominators, to varying extents. Journal Editors perceived that this occurred for various reasons, including deliberate editorial practices. The vulnerability of the IF to editorial manipulation and Editors' dissatisfaction with it as the sole measure of journal quality lend weight to the need for complementary measures."
4235332646,What is open peer review? A systematic review,"Background: “Open peer review” (OPR), despite being a major pillar of Open Science, has neither a standardized definition nor an agreed schema of its features and implementations. The literature reflects this, with a myriad of overlapping and often contradictory definitions. While the term is used by some to refer to peer review where the identities of both author and reviewer are disclosed to each other, for others it signifies systems where reviewer reports are published alongside articles. For others it signifies both of these conditions, and for yet others it describes systems where not only “invited experts” are able to comment. For still others, it includes a variety of combinations of these and other novel methods.Methods: Recognising the absence of a consensus view on what open peer review is, this article undertakes a systematic review of definitions of “open peer review” or “open review”, to create a corpus of 122 definitions. These definitions are then systematically analysed to build a coherent typology of the many different innovations in peer review signified by the term, and hence provide the precise technical definition currently lacking.Results: This quantifiable data yields rich information on the range and extent of differing definitions over time and by broad subject area. Quantifying definitions in this way allows us to accurately portray exactly how ambiguously the phrase “open peer review” has been used thus far, for the literature offers a total of 22 distinct configurations of seven traits, effectively meaning that there are 22 different definitions of OPR in the literature.Conclusions: Based on this work, I propose a pragmatic definition of open peer review as an umbrella term for a number of overlapping ways that peer review models can be adapted in line with the ethos of Open Science, including making reviewer and author identities open, publishing review reports and enabling greater participation in the peer review process."
3003960018,Altmetrics and societal impact measurements: Match or mismatch? A literature review,"Can alternative metrics (altmetrics) data be used to measure societal impact? We wrote this literature overview of empirical studies in order to find an answer to this question. The overview includes two parts. The first part, ""societal impact measurements"", explains possible methods and problems in measuring the societal impact of research, case studies for societal impact measurement, societal impact considerations at funding organizations, and the societal problems that should be solved by science. The second part of the review, ""altmetrics"", addresses a major question in research evaluation, which is whether altmetrics are proper indicators for measuring the societal impact of research. In the second part we explain the data sources used for altmetrics studies and the importance of field-normalized indicators for impact measurements. This review indicates that it should be relevant for impact measurements to be oriented towards pressing societal problems. Case studies in which societal impact of certain pieces of research is explained seem to provide a legitimate method for measuring societal impact. In the use of altmetrics, field-specific differences should be considered by applying field normalization (in cross-field comparisons). Altmetrics data such as social media counts might mainly reflect the public interest and discussion of scholarly works rather than their societal impact. Altmetrics (Twitter data) might be especially fruitfully employed for research evaluation purposes, if they are used in the context of network approaches. Conclusions based on altmetrics data in research evaluation should be drawn with caution."
2008516411,Prediction of citation counts for clinical articles at two years using data available within three weeks of publication: retrospective cohort study,"To determine if citation counts at two years could be predicted for clinical articles that pass basic criteria for critical appraisal using data within three weeks of publication from external sources and an online article rating service.Retrospective cohort study.Online rating service, Canada.1274 articles from 105 journals published from January to June 2005, randomly divided into a 60:40 split to provide derivation and validation datasets.20 article and journal features, including ratings of clinical relevance and newsworthiness, routinely collected by the McMaster online rating of evidence system, compared with citation counts at two years.The derivation analysis showed that the regression equation accounted for 60% of the variation (R2=0.60, 95% confidence interval 0.538 to 0.629). This model applied to the validation dataset gave a similar prediction (R2=0.56, 0.476 to 0.596, shrinkage 0.04; shrinkage measures how well the derived equation matches data from the validation dataset). Cited articles in the top half and top third were predicted with 83% and 61% sensitivity and 72% and 82% specificity. Higher citations were predicted by indexing in numerous databases; number of authors; abstraction in synoptic journals; clinical relevance scores; number of cited references; and original, multicentred, and therapy articles from journals with a greater proportion of articles abstracted.Citation counts can be reliably predicted at two years using data within three weeks of publication."
2085692706,Effectiveness of Journal Ranking Schemes as a Tool for Locating Information,"The rise of electronic publishing, preprint archives, blogs, and wikis is raising concerns among publishers, editors, and scientists about the present day relevance of academic journals and traditional peer review. These concerns are especially fuelled by the ability of search engines to automatically identify and sort information. It appears that academic journals can only remain relevant if acceptance of research for publication within a journal allows readers to infer immediate, reliable information on the value of that research.Here, we systematically evaluate the effectiveness of journals, through the work of editors and reviewers, at evaluating unpublished research. We find that the distribution of the number of citations to a paper published in a given journal in a specific year converges to a steady state after a journal-specific transient time, and demonstrate that in the steady state the logarithm of the number of citations has a journal-specific typical value. We then develop a model for the asymptotic number of citations accrued by papers published in a journal that closely matches the data.Our model enables us to quantify both the typical impact and the range of impacts of papers published in a journal. Finally, we propose a journal-ranking scheme that maximizes the efficiency of locating high impact research."
1966279025,A heuristic approach to author name disambiguation in bibliometrics databases for large-scale research assessments,"National exercises for the evaluation of research activity by universities are becoming regular practice in ever more countries. These exercises have mainly been conducted through the application of peer-review methods. Bibliometrics has not been able to offer a valid large-scale alternative because of almost overwhelming difficulties in identifying the true author of each publication. We will address this problem by presenting a heuristic approach to author name disambiguation in bibliometric datasets for large-scale research assessments. The application proposed concerns the Italian university system, consisting of 80 universities and a research staff of over 60,000 scientists. The key advantage of the proposed approach is the ease of implementation. The algorithms are of practical application and have considerably better scalability and expandability properties than state-of-the-art unsupervised approaches. Moreover, the performance in terms of precision and recall, which can be further improved, seems thoroughly adequate for the typical needs of large-scale bibliometric research assessments."
2894029326,Publishing and Promotion in Economics: The Tyranny of the Top Five,"This paper examines the relationship between placement of publications in top five (T5) journals and receipt of tenure in academic economics departments. Analyzing the job histories of tenure–track economists hired by the top 35 US economics departments, we find that T5 publications have a powerful influence on tenure decisions and rates of transition to tenure. A survey of the perceptions of young economists supports the formal statistical analysis. Pursuit of T5 publications has become the obsession of the next generation of economists. However, the T5 screen is far from reliable. A substantial share of influential publications appear in non-T5 outlets. Reliance on the T5 to screen talent incentivizes careerism over creativity.( JEL A14, I23, J44, J62)"
1993001003,The Eigenfactor MetricsTM: A Network Approach to Assessing Scholarly Journals,"Limited time and budgets have created a legitimate need for quantitative measures of scholarly work. The well-known journal impact factor is the leading measure of this sort; here we describe an alternative approach based on the full structure of the scholarly citation network. The Eigenfactor Metrics—Eigenfactor Score and Article Influence Score—use an iterative ranking scheme similar to Google’s PageRank algorithm. By this approach, citations from top journals are weighted more heavily than citations from lower-tier publications. Here we describe these metrics and the rankings that they provide."
2025365931,Clustering More than Two Million Biomedical Publications: Comparing the Accuracies of Nine Text-Based Similarity Approaches,"We investigate the accuracy of different similarity approaches for clustering over two million biomedical documents. Clustering large sets of text documents is important for a variety of information needs and applications such as collection management and navigation, summary and analysis. The few comparisons of clustering results from different similarity approaches have focused on small literature sets and have given conflicting results. Our study was designed to seek a robust answer to the question of which similarity approach would generate the most coherent clusters of a biomedical literature set of over two million documents.We used a corpus of 2.15 million recent (2004-2008) records from MEDLINE, and generated nine different document-document similarity matrices from information extracted from their bibliographic records, including titles, abstracts and subject headings. The nine approaches were comprised of five different analytical techniques with two data sources. The five analytical techniques are cosine similarity using term frequency-inverse document frequency vectors (tf-idf cosine), latent semantic analysis (LSA), topic modeling, and two Poisson-based language models--BM25 and PMRA (PubMed Related Articles). The two data sources were a) MeSH subject headings, and b) words from titles and abstracts. Each similarity matrix was filtered to keep the top-n highest similarities per document and then clustered using a combination of graph layout and average-link clustering. Cluster results from the nine similarity approaches were compared using (1) within-cluster textual coherence based on the Jensen-Shannon divergence, and (2) two concentration measures based on grant-to-article linkages indexed in MEDLINE.PubMed's own related article approach (PMRA) generated the most coherent and most concentrated cluster solution of the nine text-based similarity approaches tested, followed closely by the BM25 approach using titles and abstracts. Approaches using only MeSH subject headings were not competitive with those based on titles and abstracts."
2045830037,Where Are the Economists Who Publish? Publication Concentration and Rankings in Europe Based on Cumulative Publications,"We measure the past production of research articles by current members of European economics institutions. All EconLit journals are used, weighted to reflect differences in quality. Both a long (1971–2000) and a short (1996–2000) time period are considered. We also provide production indices that take into account the authors' career length. The total output of each research center is measured as well as its production per member. The focus is on 600 centers from eighteen European countries (EU 14, Israel, Norway, Switzerland, and Turkey). European centers are compared to the top sixty U.S. economics departments. Statistics regarding the concentration of article production across researchers, institutions, and countries are provided, as well as on publication habits."
2081438176,"The Assessment of Science: The Relative Merits of Post-Publication Review, the Impact Factor, and the Number of Citations","The assessment of scientific publications is an integral part of the scientific process. Here we investigate three methods of assessing the merit of a scientific paper: subjective post-publication peer review, the number of citations gained by a paper, and the impact factor of the journal in which the article was published. We investigate these methods using two datasets in which subjective post-publication assessments of scientific publications have been made by experts. We find that there are moderate, but statistically significant, correlations between assessor scores, when two assessors have rated the same paper, and between assessor score and the number of citations a paper accrues. However, we show that assessor score depends strongly on the journal in which the paper is published, and that assessors tend to over-rate papers published in journals with high impact factors. If we control for this bias, we find that the correlation between assessor scores and between assessor score and the number of citations is weak, suggesting that scientists have little ability to judge either the intrinsic merit of a paper or its likely impact. We also show that the number of citations a paper receives is an extremely error-prone measure of scientific merit. Finally, we argue that the impact factor is likely to be a poor measure of merit, since it depends on subjective assessment. We conclude that the three measures of scientific merit considered here are poor; in particular subjective assessments are an error-prone, biased, and expensive method by which to assess merit. We argue that the impact factor may be the most satisfactory of the methods we have considered, since it is a form of pre-publication review. However, we emphasise that it is likely to be a very error-prone measure of merit that is qualitative, not quantitative."
3125468813,Do Rankings Reflect Research Quality?,"Publication and citation rankings have become major indicators of the scientific worth of universities and determine to a large extent the career of individual scholars. Such rankings do not effectively measure research quality, which should be the essence of any evaluation. These quantity rankings are not objective; two citation rankings, based on different samples, produce entirely different results. For that reason, an alternative ranking is developed as a quality indicator, based on membership on academic editorial boards of professional journals. It turns out that the ranking of individual scholars based on that measure is far from objective. Furthermore, the results differ markedly, depending on whether research quantity or quality is considered. Thus, career decisions based on rankings are dominated by chance and do not reflect research quality. We suggest that evaluations should rely on multiple criteria. Public management should return to approved methods such as engaging independent experts who in turn provide measurements of research quality for their research communities."
2413426462,Impact of interventions to improve the quality of peer review of biomedical journals: a systematic review and meta-analysis,"The peer review process is a cornerstone of biomedical research. We aimed to evaluate the impact of interventions to improve the quality of peer review for biomedical publications. We performed a systematic review and meta-analysis. We searched CENTRAL, MEDLINE (PubMed), Embase, Cochrane Database of Systematic Reviews, and WHO ICTRP databases, for all randomized controlled trials (RCTs) evaluating the impact of interventions to improve the quality of peer review for biomedical publications. We selected 22 reports of randomized controlled trials, for 25 comparisons evaluating training interventions (n = 5), the addition of a statistical peer reviewer (n = 2), use of a checklist (n = 2), open peer review (i.e., peer reviewers informed that their identity would be revealed; n = 7), blinded peer review (i.e., peer reviewers blinded to author names and affiliation; n = 6) and other interventions to increase the speed of the peer review process (n = 3). Results from only seven RCTs were published since 2004. As compared with the standard peer review process, training did not improve the quality of the peer review report and use of a checklist did not improve the quality of the final manuscript. Adding a statistical peer review improved the quality of the final manuscript (standardized mean difference (SMD), 0.58; 95 % CI, 0.19 to 0.98). Open peer review improved the quality of the peer review report (SMD, 0.14; 95 % CI, 0.05 to 0.24), did not affect the time peer reviewers spent on the peer review (mean difference, 0.18; 95 % CI, –0.06 to 0.43), and decreased the rate of rejection (odds ratio, 0.56; 95 % CI, 0.33 to 0.94). Blinded peer review did not affect the quality of the peer review report or rejection rate. Interventions to increase the speed of the peer review process were too heterogeneous to allow for pooling the results. Despite the essential role of peer review, only a few interventions have been assessed in randomized controlled trials. Evidence-based peer review needs to be developed in biomedical journals."
2047000140,"Publication rate expressed by age, gender and academic position – A large-scale analysis of Norwegian academic staff","This study investigates how scientific performance in terms of publication rate is influenced by the gender, age and academic position of the researchers. Previous studies have shown that these factors are important variables when analysing scientific productivity at the individual level. What is new with our approach is that we have been able to identify the relative importance of the different factors based on regression analyses (OLS) of each major academic field. The study, involving almost 12,400 Norwegian university researchers, shows that academic position is more important than age and gender. In the fields analysed, the regression model can explain 13.5–19 per cent of the variance in the publication output at the levels of individuals. This also means that most of the variance in publication rate is due to other factors."
1971383575,Psychiatry and the Hirsch h -index: The Relationship Between Journal Impact Factors and Accrued Citations,"There is considerable debate on the use and abuse of journal impact factors and on selecting the most appropriate indicator to assess research outcome for an individual or group of scientists. Internet searches using Web of Science and Scopus were conducted to retrieve citation data for an individual in order to calculate nine variants of Hirsch'sh-index. Citations to articles published in a wide range of psychiatric journals in the periods 1995–99 and 2000–05 were analyzed using Web of Science. Comparisons were made between journal impact factor,h-index of citations from publication to 2008, and the proportion of articles cited at least 30 or 50 times. For up to 14 years post-publication, there was a strong positive relationship between journal impact factor andh-index for citations received. Journal impact factor was also compared to the percentage of articles cited at least 30 or 50 times—a comparison that showed wide variations between journals with similar impact factors. This study found that 40%–50% of the articles published in the top ten psychiatry journals ranked by impact factor acquire 30 to 50 citations within ten to fifteen years. Despite certain flaws and weaknesses, theh-index provides a better way to assess long-term performance of articles or authors than using a journal's impact factor, and it provides an alternative way to assess a journal's long-term ranking."
2525687297,Academic Research in the 21st Century: Maintaining Scientific Integrity in a Climate of Perverse Incentives and Hypercompetition,"Over the last 50 years, we argue that incentives for academic scientists have become increasingly perverse in terms of competition for research funding, development of quantitative metrics to measure performance, and a changing business model for higher education itself. Furthermore, decreased discretionary funding at the federal and state level is creating a hypercompetitive environment between government agencies (e.g., EPA, NIH, CDC), for scientists in these agencies, and for academics seeking funding from all sources—the combination of perverse incentives and decreased funding increases pressures that can lead to unethical behavior. If a critical mass of scientists become untrustworthy, a tipping point is possible in which the scientific enterprise itself becomes inherently corrupt and public trust is lost, risking a new dark age with devastating consequences to humanity. Academia and federal agencies should better support science as a public good, and incentivize altruistic and ethical outcomes, while de-emphasizing output."
3121144552,Evolving Standards for Academic Publishing: A q‐r Theory,"This paper develops models of quality standards to examine two trends: academic journals increasingly require extensive revisions of submissions, and articles are becoming longer and changing in other ways. Papers are modeled as varying along two quality dimensions: q reflects the importance of the main ideas and r other aspects of quality. Observed trends are regarded as increases in r‐quality. A static equilibrium model illustrates comparative statics explanations. A dynamic model in which referees (with a biased view of their own work) learn social norms for weighting q and r is shown to produce a long, gradual evolution of social norms."
2011908708,Modifying publication practices in response to funding formulas,"Journal Article Modifying publication practices in response to funding formulas Get access Linda Butler Linda Butler Search for other works by this author on: Oxford Academic Google Scholar Research Evaluation, Volume 12, Issue 1, April 2003, Pages 39–46, https://doi.org/10.3152/147154403781776780 Published: 01 April 2003"
1603147783,The hand of science: academic writing and its rewards,Part 1 Acknowledgments Chapter 2 1 Scholars and Scripts Chapter 3 2 Epistemic Cultures Chapter 4 3 Hyperauthorship Chapter 5 4 Information Space Chapter 6 5 Intellectual Collaboration Chapter 7 6 The Reward System Chapter 8 7 Symbolic Capitalism Chapter 9 8 The Attention Economy Chapter 10 9 Scientometric Spectroscopy Part 11 Index Part 12 About the Author
2744367301,What do we know about grant peer review in the health sciences?,"Background: Peer review decisions award an estimated >95% of academic medical research funding, so it is crucial to understand how well they work and if they could be improved. Methods: This paper summarises evidence from 105 papers identified through a literature search on the effectiveness and burden of peer review for grant funding. Results: There is a remarkable paucity of evidence about the efficiency of peer review for funding allocation, given its centrality to the modern system of science. From the available evidence, we can identify some conclusions around the effectiveness and burden of peer review. The strongest evidence around effectiveness indicates a bias against innovative research. There is also fairly clear evidence that peer review is, at best, a weak predictor of future research performance, and that ratings vary considerably between reviewers. There is some evidence of age bias and cronyism. Good evidence shows that the burden of peer review is high and that around 75% of it falls on applicants. By contrast, many of the efforts to reduce burden are focused on funders and reviewers/panel members. Conclusions: We suggest funders should acknowledge, assess and analyse the uncertainty around peer review, even using reviewers’ uncertainty as an input to funding decisions. Funders could consider a lottery element in some parts of their funding allocation process, to reduce both burden and bias, and allow better evaluation of decision processes. Alternatively, the distribution of scores from different reviewers could be better utilised as a possible way to identify novel, innovative research. Above all, there is a need for open, transparent experimentation and evaluation of different ways to fund research. This also requires more openness across the wider scientific community to support such investigations, acknowledging the lack of evidence about the primacy of the current system and the impossibility of achieving perfection."
2032020071,Conservatism and risk-taking in peer review: Emerging ERC practices,"Journal Article Conservatism and risk-taking in peer review: Emerging ERC practices Get access Terttu Luukkonen Terttu Luukkonen * The Research Institute of the Finnish Economy, Lönnrotinkatu 4 B, FIN-00120 Helsinki *Corresponding author. Email: terttu.luukkonen@etla.fi. Search for other works by this author on: Oxford Academic Google Scholar Research Evaluation, Volume 21, Issue 1, March 2012, Pages 48–60, https://doi.org/10.1093/reseval/rvs001 Published: 15 February 2012"
2324810750,"ResearchGate articles: Age, discipline, audience size, and impact","The large multidisciplinary academic social website ResearchGate aims to help academics to connect with each other and to publicize their work. Despite its popularity, little is known about the age and discipline of the articles uploaded and viewed in the site and whether publication statistics from the site could be useful impact indicators. In response, this article assesses samples of ResearchGate articles uploaded at specific dates, comparing their views in the site to their Mendeley readers and Scopus‐indexed citations. This analysis shows that ResearchGate is dominated by recent articles, which attract about three times as many views as older articles. ResearchGate has uneven coverage of scholarship, with the arts and humanities, health professions, and decision sciences poorly represented and some fields receiving twice as many views per article as others. View counts for uploaded articles have low to moderate positive correlations with both Scopus citations and Mendeley readers, which is consistent with them tending to reflect a wider audience than Scopus‐publishing scholars. Hence, for articles uploaded to the site, view counts may give a genuinely new audience indicator."
2140811105,"Accountability in context: effects of research evaluation systems on publication practices, disciplinary norms, and individual working routines in the faculty of Arts at Uppsala University","Journal Article Accountability in context: effects of research evaluation systems on publication practices, disciplinary norms, and individual working routines in the faculty of Arts at Uppsala University Get access Björn Hammarfelt, Björn Hammarfelt * 1Department of ALM, Uppsala University, 751 05 Uppsala, Sweden and 2CWTS, Leiden University, 2333 AL Leiden, The Netherlands *Corresponding author. Email: bjorn.hammarfelt@abm.uu.se Search for other works by this author on: Oxford Academic Google Scholar Sarah de Rijcke Sarah de Rijcke 1Department of ALM, Uppsala University, 751 05 Uppsala, Sweden and 2CWTS, Leiden University, 2333 AL Leiden, The Netherlands Search for other works by this author on: Oxford Academic Google Scholar Research Evaluation, Volume 24, Issue 1, January 2015, Pages 63–77, https://doi.org/10.1093/reseval/rvu029 Published: 10 December 2014"
2135336774,The level of non-citation of articles within a journal as a measure of quality: a comparison to the impact factor,"Current methods of measuring the quality of journals assume that citations of articles within journals are normally distributed. Furthermore using journal impact factors to measure the quality of individual articles is flawed if citations are not uniformly spread between articles. The aim of this study was to assess the distribution of citations to articles and use the level of non-citation of articles within a journal as a measure of quality. This ranking method is compared with the impact factor, as calculated by ISI(R).Total citations gained by October 2003, for every original article and review published in current immunology (13125 articles; 105 journals) and surgical (17083 articles; 120 journals) fields during 2001 were collected using ISI(R) Web of Science.The distribution of citation of articles within an individual journal is mainly non-parametric throughout the literature. One sixth (16.7%; IQR 13.6-19.2) of articles in a journal accrue half the total number of citations to that journal. There was a broader distribution of citation to articles in higher impact journals and in the field of immunology compared to surgery. 23.7% (IQR 14.6-42.4) of articles had not yet been cited. Levels of non-citation varied between journals and subject fields. There was a significant negative correlation between the proportion of articles never cited and a journal's impact factor for both immunology (rho = -0.854) and surgery journals (rho = -0.924).Ranking journals by impact factor and non-citation produces similar results. Using a non-citation rate is advantageous as it creates a clear distinction between how citation analysis is used to determine the quality of a journal (low level of non-citation) and an individual article (citation counting). Non-citation levels should therefore be made available for all journals."
2077217220,Bibliometrics and beyond: some thoughts on web-based citation analysis,"The idea of a unified citation index to the literature of science was first outlined by Eugene Garfield [1] in 1955 in the journal Science. Science Citation Index has since established itself as the gold standard for scientific information retrieval. It has also become the database of choice for citation analysts and evaluative bibliometricians worldwide. As scientific publication moves to the web, and novel approaches to scholarly communication and peer review establish themselves, new methods of citation and link analysis will emerge to capture often liminal expressions of peer esteem, influence and approbation. The web thus affords bibliometricians rich opportunities to apply and adapt their techniques to new contexts and content: the age of ‘bibliometric spectroscopy’ [2] is dawning."
2077611838,Highly cited works in neurosurgery. Part I: the 100 top-cited papers in neurosurgical journals,"The number of citations a published article receives is a measure of its impact in the scientific community. This study identifies and characterizes the current 100 top-cited articles in journals specifically dedicated to neurosurgery.Neurosurgical journals were identified using the Institute for Scientific Information Journal Citation Reports. A search was performed using Institute for Scientific Information Web of Science for articles appearing in each of these journals. The 100 top-cited articles were selected and analyzed.The 100 most cited manuscripts in neurosurgical journals appeared in 3 of 13 journals dedicated to neurosurgery. These included 79 in the Journal of Neurosurgery, 11 in the Journal of Neurology, Neurosurgery and Psychiatry, and 10 in Neurosurgery. The individual citation counts for these articles ranged from 287 to 1515. Seventy-seven percent of articles were published between 1976 and 1995. Representation varied widely across neurosurgical disciplines, with cerebrovascular diseases leading (43 articles), followed by trauma (27 articles), stereotactic and functional neurosurgery (13 articles), and neurooncology (12 articles). The study types included 5 randomized trials, 5 cooperative studies, 1 observational cohort study, 69 case series, 8 review articles, and 12 animal studies. Thirty articles dealt with surgical management and 12 with nonsurgical management. There were 15 studies of natural history of disease or outcomes after trauma, 11 classification or grading scales, and 10 studies of human pathophysiology.The most cited articles in neurosurgical journals are trials evaluating surgical or medical therapies, descriptions of novel techniques, or systems for classifying or grading disease. The time of publication, field of study, nature of the work, and the journal in which the work appears are possible determinants of the likelihood of citation and impact."
2141967394,The influence of management journals in the 1980s and 1990s,"It is difficult to get a clear picture of the relative influence of management journals because previous studies have focused on a single sub-area in the field over a relatively restricted number of years, and/or have used inconsistent criteria to judge journal influence. Therefore, the purpose of this study is to examine journal influence using citations from 28 journals over the past two decades. The findings show that the top seven journals accounted for 61 percent of all of the citations in the journals included, and that the three journals that showed the greatest increase in influence over the past 20 years were AMJ, AMR, and SMJ. Copyright © 2005 John Wiley & Sons, Ltd."
2123730213,Bibliometric studies on single journals: a review,"This paper covers a total of 82 bibliometric studies on single journals (62 studies cover unique titles) published between 1998 and 2008 grouped into the following fields; Arts, Humanities and Social Sciences (12 items); Medical and Health Sciences (19 items); Sciences and Technology (30 items) and Library and Information Sciences (21 items). Under each field the studies are described in accordance to their geographical location in the following order, United Kingdom, United States and Americana, Europe, Asia (India, Africa and Malaysia). For each study, elements described are (a) the journal’s publication characteristics and indexation information; (b) the objectives; (c) the sampling and bibliometric measures used; and (d) the results observed. A list of journal titles studied is appended. The results show that (a)bibliometric studies cover journals in various fields; (b) there are several revisits of some journals which are considered important; (c) Asian and African contributions is high (41.4% of total studies; 43.5% covering unique titles), United States (30.4% of total; 31.0% on unique titles), Europe (18.2% of total and 14.5% on unique titles) and the United Kingdom (10% of total and 11% on unique titles); (d) a high number of bibliometrists are Indians and as such coverage of Indian journals is high (28% of total studies; 30.6% of unique titles); and (e) the quality of the journals and their importance either nationally or internationally are inferred from their indexation status."
1991869139,The Number of Scholarly Documents on the Public Web,"The number of scholarly documents available on the web is estimated using capture/recapture methods by studying the coverage of two major academic search engines: Google Scholar and Microsoft Academic Search. Our estimates show that at least 114 million English-language scholarly documents are accessible on the web, of which Google Scholar has nearly 100 million. Of these, we estimate that at least 27 million (24%) are freely available since they do not require a subscription or payment of any kind. In addition, at a finer scale, we also estimate the number of scholarly documents on the web for fifteen fields: Agricultural Science, Arts and Humanities, Biology, Chemistry, Computer Science, Economics and Business, Engineering, Environmental Sciences, Geosciences, Material Science, Mathematics, Medicine, Physics, Social Sciences, and Multidisciplinary, as defined by Microsoft Academic Search. In addition, we show that among these fields the percentage of documents defined as freely available varies significantly, i.e., from 12 to 50%."
1968612073,Six Decades of Top Economics Publishing: Who and How?,"Presenting data on all full-length articles in the three top general economics journals for one year in each decade 1960s–2010s, I analyze changes in patterns of coauthorship, age structure and methodology, and their possible causes. The distribution of number of authors has shifted steadily rightward. In the last two decades, the fraction of older authors has almost quadrupled. Top journals are publishing many fewer papers that represent pure theory, regardless of subfield, somewhat less empirical work based on publicly available data sets, and many more empirical studies based on data collected by the author(s) or on laboratory or field experiments. (JEL A14)"
2115331617,The White Bull effect: abusive coauthorship and publication parasitism,"Junior researchers can be abused and bullied by unscrupulous senior collaborators. This article describes the profile of a type of serial abuser, the White Bull, who uses his academic seniority to distort authorship credit and who disguises his parasitism with carefully premeditated deception. Further research into the personality traits of such perpetrators is warranted."
2167304379,Self‐citation and self‐reference: Credibility and promotion in academic publication,"Abstract Author self‐citation has long been of interest to those working in informetrics for what it reveals about the publishing behavior of individuals and their relationships within academic networks. While this research has produced interesting insights, it typically assumes either that self‐citation is a neutral form of reporting not unlike references to others' work or an unsavory kind of academic egotism. By examining self‐citation in a wider context of self‐mention, however, the phenomenon can be seen as part of a more comprehensive rhetorical strategy for emphasizing a writer's personal contribution to a piece of research and strengthening his or her knowledge claims, research credibility, and wider standing in the discipline. These meanings are not easily revealed through quantitative bibliometric methods and require careful text analyses and discourse‐based interviews with academics. In this paper I explore the use of self‐citation and authorial mention in a corpus of 240 research articles and 800 abstracts in eight disciplines. Through an analysis of these texts and interviews with expert informants I show how self‐mention is used and the ways these uses reflect both the promotional strategies of individuals and the epistemological practices of their disciplines."
772173499,Accounting for Impact? The Journal Impact Factor and the Making of Biomedical Research in the Netherlands,"The range and types of performance metrics has recently proliferated in academic settings, with bibliometric indicators being particularly visible examples. One field that has traditionally been hospitable towards such indicators is biomedicine. Here the relative merits of bibliometrics are widely discussed, with debates often portraying them as heroes or villains. Despite a plethora of controversies, one of the most widely used indicators in this field is said to be the Journal Impact Factor (JIF). In this article we argue that much of the current debates around researchers' uses of the JIF in biomedicine can be classed as 'folk theories': explanatory accounts told among a community that seldom (if ever) get systematically checked. Such accounts rarely disclose how knowledge production itself becomes more-or-less consolidated around the JIF. Using ethnographic materials from different research sites in Dutch University Medical Centers, this article sheds new empirical and theoretical light on how performance metrics variously shape biomedical research on the 'shop floor.' Our detailed analysis underscores a need for further research into the constitutive effects of evaluative metrics."
1505519009,Journal Evaluation: Technical and Practical Issues,"This essay provides an overview of journal evaluation indicators. It highlights the strengths and weaknesses of different indicators, together with their range of applicability. The definition of a quality journal, different notions of impact factors, the meaning of ranking journals, and possible biases in citation databases are also discussed. Attention is given to using the journal impact in evaluation studies. The quality of a journal is a multifaceted notion. Journals can be evaluated for different purposes, and hence the results of such evaluation exercises can be quite different depending on the indicator(s) used. The impact factor, in one of its versions, is probably the most used indicator when it comes to gauging the visibility of a journal on the research front. Generalized impact factors, over periods longer than the traditional two years, are better indicators for the long-term value of a journal. As with all evaluation studies, care must be exercised when considering journal impact factors as a quality indicator. It seems best to use a whole battery of indicators (including several impact factors) and to change this group of indicators depending on the purpose of the evaluation study. Nowadays it goes without saying that special attention is paid to ejournals and specific indicators for this type of journal."
2130548493,Why do we need algorithmic historiography?,"Abstract This article discusses the rationale for creating historiographs of scholarly topics using a new program called HistCite ™, which produces a variety of analyses to aid the historian identify key events (papers), people (authors), and journals in a field. By creating a genealogic profile of the evolution, the program aids the scholar in evaluating the paradigm involved."
2015134878,Relation between online “hit counts” and subsequent citations: prospective study of research papers in the BMJ,"Evaluation of published medical research remains a challenge. Two classic yardsticks are the citation count (the number of times a given paper is cited by others)1 2 and the impact factor of the journal that published the paper (which reflects the average number of citations per article).2 3 However, the citation count can be assessed only several years after publication, and the impact factor is not paper specific and is thus virtually meaningless in assessing any given paper.3 Another measure, which can be obtained rapidly and is paper specific, is the “hit count” (the number of times a paper is accessed online). Whether this count predicts citations is unknown. I examined this issue prospectively in a cohort of papers published in the BMJ .The study used articles published in volume 318 of the BMJ (1999) in sections titled Papers, General Practice, and Information in Practice. The hit counts (full text articles, HTML version) for the main body of each article within a week of publication were provided by a …"
2802794772,"Researcher and Author Impact Metrics: Variety, Value, and Context","Numerous quantitative indicators are currently available for evaluating research productivity.No single metric is suitable for comprehensive evaluation of the author-level impact.The choice of particular metrics depends on the purpose and context of the evaluation.The aim of this article is to overview some of the widely employed author impact metrics and highlight perspectives of their optimal use.The h-index is one of the most popular metrics for research evaluation, which is easy to calculate and understandable for non-experts.It is automatically displayed on researcher and author profiles on citation databases such as Scopus and Web of Science.Its main advantage relates to the combined approach to the quantification of publication and citation counts.This index is increasingly cited globally.Being an appropriate indicator of publication and citation activity of highly productive and successfully promoted authors, the h-index has been criticized primarily for disadvantaging early career researchers and authors with a few indexed publications.Numerous variants of the index have been proposed to overcome its limitations.Alternative metrics have also emerged to highlight 'societal impact.'However, each of these traditional and alternative metrics has its own drawbacks, necessitating careful analyses of the context of social attention and value of publication and citation sets.Perspectives of the optimal use of researcher and author metrics is dependent on evaluation purposes and compounded by information sourced from various global, national, and specialist bibliographic databases."
1868442603,Open Access Overview,"This article is an introduction to open access. It was originally published online on June 21, 2004. The original is available at http://www.earlham.edu/~peters/fos/overview.htm . This work is licensed under a Creative Commons Attribution 2.5 License ."
2949884440,Follow the leader: On the relationship between leadership and scholarly impact in international collaborations,"National contributions to science are influenced by a number of factors, including economic capacity, national scientific priorities, science policy, and institutional settings and cultures. Nations do not have equal opportunities to access the global scientific market, and therefore, often seek out international partners with complementary resources and expertise. This study aims at investigating national collaboration strategies, with a focus on research leadership-measured through corresponding authorship-and its relationship with scientific impact. Results show that countries with higher R&D investments are more scientifically independent, and confirm that international collaboration is positively related to citation impact. However, leadership in international collaboration is inversely related with a countries' share of international collaboration and there is a very little relationship between citation impact and international leadership. For instance, most countries-and particularly those that have fewer resources-have higher scientific impact when they are not leading. This suggests that, despite increasing global participation in science, most international collaborations are asymmetrical, and that the research system remains structured around a few dominate nations."
2145203895,Eminence of scientists in the light of the h-index and other scientometric indicators,"Scientometrics cannot offer a simple consistent method for measuring the scientific eminence of individuals. The h-index method introduced by Hirsch was found applicable for evaluating publications of senior scientists with similar publishing features, only. Some simple methods — using the number of citations and journal papers, and the number of citations obtained by the most frequently cited papers — are suggested and tested to demonstrate the advantages and disadvantages of such indexes. The results indicate that calculating scientometric indexes for individuals, self-citations should be excluded and the effect of the different bibliometric features of the field should be taken into account. The correctness of the indexes used for evaluating journal papers of individuals should be investigated also on the individual level."
2592989848,Duration and quality of the peer review process: the author’s perspective,"To gain insight into the duration and quality of the scientific peer review process, we analyzed data from 3500 review experiences submitted by authors to the SciRev.sc website. Aspects studied are duration of the first review round, total review duration, immediate rejection time, the number, quality, and difficulty of referee reports, the time it takes authors to revise and resubmit their manuscript, and overall quality of the experience. We find clear differences in these aspects between scientific fields, with Medicine, Public health, and Natural sciences showing the shortest durations and Mathematics and Computer sciences, Social sciences, Economics and Business, and Humanities the longest. One-third of journals take more than 2 weeks for an immediate (desk) rejection and one sixth even more than 4 weeks. This suggests that besides the time reviewers take, inefficient editorial processes also play an important role. As might be expected, shorter peer review processes and those of accepted papers are rated more positively by authors. More surprising is that peer review processes in the fields linked to long processes are rated highest and those in the fields linked to short processes lowest. Hence authors' satisfaction is apparently influenced by their expectations regarding what is common in their field. Qualitative information provided by the authors indicates that editors can enhance author satisfaction by taking an independent position vis-à-vis reviewers and by communicating well with authors."
1997871319,"Beyond authorship: attribution, contribution, collaboration, and credit","Key points As the number of authors on scientific publications increases, ordered lists of author names are proving inadequate for the purposes of attribution and credit. A multi‐stakeholder group has produced a contributor role taxonomy for use in scientific publications. Identifying specific contributions to published research will lead to appropriate credit, fewer author disputes, and fewer disincentives to collaboration and the sharing of data and code."
4252643264,Problems of citation analysis: A study of uncited and seldom‐cited influences,"Abstract To determine influences on the production of a scientific article, the content of the article must be studied. We examined articles in biogeography and found that most of the influence is not cited, specific types of articles that are influential are cited while other types of that also are influential are not cited, and work that is “uncited” and “seldom cited” is used extensively. As a result, evaluative citation analysis should take uncited work into account."
2806012053,Twenty years of theJournal of Knowledge Management: a bibliometric analysis,"Purpose In 2017, the Journal of Knowledge Management (JKM) celebrates its 20th anniversary. This study aims to show an updated analysis of their publications to provide a general overview of the journal, focusing on a bibliometric analysis of its publications between 1997 and 2016. Design/methodology/approach The methodology involves two procedures: a performance analysis and a science mapping analysis of JKM. The performance analysis uses a series of bibliometric indicators such as h -index, productivity and citations. This analysis considers different dimensions, including papers, authors, universities and countries. VOSviewer software is used to carry out the mapping of science of JKM, which, based on the concurrence of key words and co-citation points of view, seeks to graphically analyze the structure of the references of this journal. Findings There is a positive evolution in the number of publications (although with certain oscillations), which shows a growing interest in publishing in JKM. The USA and the UK lead the publications in this journal, although at a regional level, Europe is the most productive. The low participation of emerging economies in JKM is also observed. Practical implications The paper will identify the leading trends in the journal in terms of papers, authors, institutions, countries, journals and keywords. This study is useful for obtaining a quick snapshot of what is happening in the journal. Originality/value From the historical record of JKM publications, this study presents an exclusive bibliometric analysis of its publications until 2016 and identifies its main trends."
2097914836,Detecting the historical roots of research fields by reference publication year spectroscopy (RPYS),"We introduce the quantitative method named “Reference Publication Year Spectroscopy” ( RPYS ). With this method one can determine the historical roots of research fields and quantify their impact on current research. RPYS is based on the analysis of the frequency with which references are cited in the publications of a specific research field in terms of the publication years of these cited references. The origins show up in the form of more or less pronounced peaks mostly caused by individual publications that are cited particularly frequently. In this study, we use research on graphene and on solar cells to illustrate how RPYS functions, and what results it can deliver."
2095972207,Clickstream Data Yields High-Resolution Maps of Science,"Background Intricate maps of science have been created from citation data to visualize the structure of scientific activity. However, most scientific publications are now accessed online. Scholarly web portals record detailed log data at a scale that exceeds the number of all existing citations combined. Such log data is recorded immediately upon publication and keeps track of the sequences of user requests (clickstreams) that are issued by a variety of users across many different domains. Given these advantages of log datasets over citation data, we investigate whether they can produce high-resolution, more current maps of science. Methodology Over the course of 2007 and 2008, we collected nearly 1 billion user interactions recorded by the scholarly web portals of some of the most significant publishers, aggregators and institutional consortia. The resulting reference data set covers a significant part of world-wide use of scholarly web portals in 2006, and provides a balanced coverage of the humanities, social sciences, and natural sciences. A journal clickstream model, i.e. a first-order Markov chain, was extracted from the sequences of user interactions in the logs. The clickstream model was validated by comparing it to the Getty Research Institute's Architecture and Art Thesaurus. The resulting model was visualized as a journal network that outlines the relationships between various scientific domains and clarifies the connection of the social sciences and humanities to the natural sciences. Conclusions Maps of science resulting from large-scale clickstream data provide a detailed, contemporary view of scientific activity and correct the underrepresentation of the social sciences and humanities that is commonly found in citation data."
3000895385,Web of Science as a data source for research on scientific and scholarly activity,"Web of Science (WoS) is the world’s oldest, most widely used and authoritative database of research publications and citations. Based on the Science Citation Index, founded by Eugene Garfield in 1964, it has expanded its selective, balanced, and complete coverage of the world’s leading research to cover around 34,000 journals today. A wide range of use cases are supported by WoS from daily search and discovery by researchers worldwide through to the supply of analytical data sets and the provision of specialized access to raw data for bibliometric partners. A long- and well-established network of such partners enables the Institute for Scientific Information (ISI) to continue to work closely with bibliometric groups around the world to the benefit of both the community and the services that the company provides to researchers and analysts."
2949684159,Early coauthorship with top scientists predicts success in academic careers,"We quantify the long term impact that the coauthorship with established top-cited scientists has on the career of junior researchers in four different scientific disciplines. Through matched pair analysis, we find that junior researchers who coauthor work with top scientists enjoy a persistent competitive advantage throughout the rest of their careers with respect to peers with similar early career profiles. Such a competitive advantage materialises as a higher probability of repeatedly coauthoring work with top-cited scientists, and, ultimately, as a higher probability of becoming one. Notably, we find that the coauthorship with a top scientist has the strongest impact on the careers of junior researchers affiliated with less prestigious institutions. As a consequence, we argue that such institutions may hold vast amounts of untapped potential, which may be realised by improving access to top scientists."
2417800084,“Excellence R Us”: university research and the fetishisation of excellence,"Abstract The rhetoric of “excellence” is pervasive across the academy. It is used to refer to research outputs as well as researchers, theory and education, individuals and organizations, from art history to zoology. But does “excellence” actually mean anything? Does this pervasive narrative of “excellence” do any good? Drawing on a range of sources we interrogate “excellence” as a concept and find that it has no intrinsic meaning in academia. Rather it functions as a linguistic interchange mechanism. To investigate whether this linguistic function is useful we examine how the rhetoric of excellence combines with narratives of scarcity and competition to show that the hyper-competition that arises from the performance of “excellence” is completely at odds with the qualities of good research. We trace the roots of issues in reproducibility, fraud, and homophily to this rhetoric. But we also show that this rhetoric is an internal, and not primarily an external, imposition. We conclude by proposing an alternative rhetoric based on soundness and capacity-building. In the final analysis, it turns out that that “excellence” is not excellent. Used in its current unqualified form it is a pernicious and dangerous rhetoric that undermines the very foundations of good research and scholarship. This article is published as part of a collection on the future of research assessment."
2148434025,"Quantifying the impact of weak, strong, and super ties in scientific careers","Scientists are frequently faced with the important decision to start or terminate a creative partnership. This process can be influenced by strategic motivations, as early career researchers are pursuers, whereas senior researchers are typically attractors, of new collaborative opportunities. Focusing on the longitudinal aspects of scientific collaboration, we analyzed 473 collaboration profiles using an ego-centric perspective which accounts for researcher-specific characteristics and provides insight into a range of topics, from career achievement and sustainability to team dynamics and efficiency. From more than 166,000 collaboration records, we quantify the frequency distributions of collaboration duration and tie-strength, showing that collaboration networks are dominated by weak ties characterized by high turnover rates. We use analytic extreme-value thresholds to identify a new class of indispensable `super ties', the strongest of which commonly exhibit >50% publication overlap with the central scientist. The prevalence of super ties suggests that they arise from career strategies based upon cost, risk, and reward sharing and complementary skill matching. We then use a combination of descriptive and panel regression methods to compare the subset of publications coauthored with a super tie to the subset without one, controlling for pertinent features such as career age, prestige, team size, and prior group experience. We find that super ties contribute to above-average productivity and a 17% citation increase per publication, thus identifying these partnerships - the analog of life partners - as a major factor in science career development."
2066364419,"Visible, less visible, and invisible work: Patterns of collaboration in 20th century chemistry","Abstract We chronicle the use of acknowledgments in 20th century chemistry by analyzing and classifying over 2,000 specimens covering a 100‐year period. Our results show that acknowledgment has gradually established itself as a constitutive element of academic writing—one that provides a revealing insight into the structural nature of subauthorship collaboration in science. Complementary data on rates of coauthorship are also presented to highlight the growing importance of teamwork and the increasing division of labor in contemporary chemistry. The results of this study are compared with the findings of a parallel study of collaboration in both the social sciences and the humanities."
2900201388,Google Scholar to overshadow them all? Comparing the sizes of 12 academic search engines and bibliographic databases,"Information on the size of academic search engines and bibliographic databases (ASEBDs) is often outdated or entirely unavailable. Hence, it is difficult to assess the scope of specific databases, such as Google Scholar. While scientometric studies have estimated ASEBD sizes before, the methods employed were able to compare only a few databases. Consequently, there is no up-to-date comparative information on the sizes of popular ASEBDs. This study aims to fill this blind spot by providing a comparative picture of 12 of the most commonly used ASEBDs. In doing so, we build on and refine previous scientometric research by counting query hit data as an indicator of the number of accessible records. Iterative query optimization makes it possible to identify a maximum number of hits for most ASEBDs. The results were validated in terms of their capacity to assess database size by comparing them with official information on database sizes or previous scientometric studies. The queries used here are replicable, so size information can be updated quickly. The findings provide first-time size estimates of ProQuest and EbscoHost and indicate that Google Scholar's size might have been underestimated so far by more than 50%. By our estimation Google Scholar, with 389 million records, is currently the most comprehensive academic search engine."
2797204405,Do altmetrics correlate with the quality of papers? A large-scale empirical study based on F1000Prime data,"In this study, we address the question whether (and to what extent, respectively) altmetrics are related to the scientific quality of papers (as measured by peer assessments). Only a few studies have previously investigated the relationship between altmetrics and assessments by peers. In the first step, we analyse the underlying dimensions of measurement for traditional metrics (citation counts) and altmetrics–by using principal component analysis (PCA) and factor analysis (FA). In the second step, we test the relationship between the dimensions and quality of papers (as measured by the post-publication peer-review system of F1000Prime assessments)–using regression analysis. The results of the PCA and FA show that altmetrics operate along different dimensions, whereas Mendeley counts are related to citation counts, and tweets form a separate dimension. The results of the regression analysis indicate that citation-based metrics and readership counts are significantly more related to quality, than tweets. This result on the one hand questions the use of Twitter counts for research evaluation purposes and on the other hand indicates potential use of Mendeley reader counts."
3099052861,Author-Reviewer Homophily in Peer Review,"Abstract The fairness of scholarly peer review has been challenged by evidence of disparities in publication outcomes based on author demographic characteristics. To assess this, we conducted an exploratory analysis of peer review outcomes of 23,876 initial submissions and 7,192 full submissions that were submitted to the biosciences journal eLife between 2012 and 2017. Women and authors from nations outside of North America and Europe were underrepresented both as gatekeepers (editors and peer reviewers) and authors. We found evidence of a homophilic relationship between the demographics of the gatekeepers and authors and the outcome of peer review; that is, there were higher rates of acceptance in the case of gender and country homophily. The acceptance rate for manuscripts with male last authors was seven percent, or 3.5 percentage points, greater than for female last authors (95% CI = [0.5, 6.4]); this gender inequity was greatest, at nine percent or about 4.8 percentage points (95% CI = [0.3, 9.1]), when the team of reviewers was all male; this difference was smaller and not significantly different for mixed-gender reviewer teams. Homogeny between countries of the gatekeeper and the corresponding author was also associated with higher acceptance rates for many countries. To test for the persistence of these effects after controlling for potentially confounding variables, we conducted a logistic regression including document and author metadata. Disparities in acceptance rates associated with gender and country of affiliation and the homophilic associations remained. We conclude with a discussion of mechanisms that could contribute to this effect, directions for future research, and policy implications. Code and anonymized data have been made available at https://github.com/murrayds/elife-analysis Author summary Peer review, the primary method by which scientific work is evaluated, is ideally a fair and equitable process in which scientific work is judged solely on its own merit. However, the integrity of peer review has been called into question based on evidence that outcomes often differ between male and female authors, and for authors in different countries. We investigated such disparities at the biosciences journal eLife by analyzing the demographics of authors and gatekeepers (editors and peer reviewers), and peer review outcomes of all submissions between 2012 and 2017. Outcomes were more favorable for male authors and those affiliated with institutions in North America and Europe; these groups were also over-represented among gatekeepers. There was evidence that peer review outcomes were influenced by homophily —a preference of gatekeepers for manuscripts from authors with shared characteristics. We discuss mechanisms that could contribute to this effect, directions for future research, and policy implications."
2015934453,The peer‐review process,"ABSTRACT The recent literature about peer review of scholarly articles is reviewed, with particular emphasis on the cost of the peer‐review process. Possible impacts of electronic scholarly publishing upon peer reviewing are discussed. Opinion among academics in their roles as authors, editors and referees seems likely to insist upon preservation of a pre‐publication refereeing system in most disciplines. As the administration of any such system seems to have a cost of about $400 per published article, any scholarly publishing system will need to locate financial support to at least that extent, and a system of lump‐sum payment by the authors' funders is best placed to cover this cost while providing universal free access to scholarly material."
2019600199,P‐Rank: An indicator measuring prestige in heterogeneous scholarly networks,"Abstract Ranking scientific productivity and prestige are often limited to homogeneous networks. These networks are unable to account for the multiple factors that constitute the scholarly communication and reward system. This study proposes a new informetric indicator, P‐Rank, for measuring prestige in heterogeneous scholarly networks containing articles, authors, and journals. P‐Rank differentiates the weight of each citation based on its citing papers, citing journals, and citing authors. Articles from 16 representative library and information science journals are selected as the dataset. Principle Component Analysis is conducted to examine the relationship between P‐Rank and other bibliometric indicators. We also compare the correlation and rank variances between citation counts and P‐Rank scores. This work provides a new approach to examining prestige in scholarly communication networks in a more comprehensive and nuanced way."
2910772330,Scholarly reputation building in the digital age: an activity-specific approach. Review article,"Seeking to understand how today's scholars may, indeed should go about building, maintaining and showcasing their professional reputation, the literature review presented here explores the reputational opportunities available to them in the increasingly open-values based, digital and networked environment of Science 2.0. Using a conceptual framework developed with the help of the European Commission and specifically designed for conducting analytical reviews and audits of the reputational value of scholarly activities, this study, examines in some detail the practices –more than 30 of them- that comprise the present-day scientific undertaking from a reputation-accruing angle."
2140557408,What's in a Number? Issues in Providing Evidence of Impact and Quality of Research(ers),"One of the challenges facing qualitative researchers in a climate in which audit culture has permeated many facets of the institutions in which they research is how to establish the impact and quality of their research. When examining track records, granting institutions place significant emphasis on publication performance. Although the quality and impact of publications have traditionally been assessed by peer review, there is currently a global trend toward the development, refinement, and increased use of quantitative metrics, particularly citation analysis and journal impact factor. In this article, the authors share their experience of using the metrics citation analysis and journal impact factor in the preparation of an application for funding. Their aim is twofold: to raise awareness about potential issues in the practical application of these metrics; and to offer critique about and, they hope, ""quality"" to the writing and rhetoric concerning how to measure publication impact and quality."
2121387070,The Manuscript Review Process,"Drawing on a 28-item survey, this article reports the editorial experiences of 173 lead authors of articles published in theAcademy of Management Journal andAcademy of Management Review, over the period 1999 to 2001, to explore some relatively new dynamics that have changed the character of the manuscript review process and given rise to a mounting debate over the proper roles of authors, referees, and editors. Among the survey’s more disturbing findings, more than one third of the responding authors reported that recommended revisions in their manuscripts were based on an editor’s or referee’s personal preferences, and almost 25% indicated that in revising their manuscripts they had actually made changes they felt were incorrect."
2049213211,"Career on the Move: Geography, Stratification and Scientific Impact","Changing institutions is an integral part of an academic life. Yet little is known about the mobility patterns of scientists at an institutional level and how these career choices affect scientific outcomes. Here, we examine over 420,000 papers, to track the affiliation information of individual scientists, allowing us to reconstruct their career trajectories over decades. We find that career movements are not only temporally and spatially localized, but also characterized by a high degree of stratification in institutional ranking. When cross-group movement occurs, we find that while going from elite to lower-rank institutions on average associates with modest decrease in scientific performance, transitioning into elite institutions does not result in subsequent performance gain. These results offer empirical evidence on institutional level career choices and movements and have potential implications for science policy."
3003345949,Altmetrics data providers: A meta-analysis review of the coverage of metrics and publication,"The aim of this paper is to review the current and most relevant literature on the use of altmetric providers since 2012. This review is supported by a meta-analysis of the coverage and metric counts obtained by more than 100 publications that have used these bibliographic platforms for altmetric studies. The article is the most comprehensive analysis of altmetric data providers (Lagotto, Altmetric.com, ImpactStory, Mendeley, PlumX, Crossref Event Data) and explores the coverage of publications, social media and events from a longitudinal view. Disciplinary differences were also analysed. The results show that most of the studies are based on Altmetric.com data. This provider is the service that captures most mentions from social media sites, blogs and news outlets. PlumX has better coverage, counting more Mendeley readers, but capturing fewer events. CED has a special coverage of mentions from Wikipedia, while Lagotto and ImpactStory are becoming disused products because of their limited reach."
2105822516,Analyzing the Scientific Evolution of Social Work Using Science Mapping,"Objectives: This article reports the first science mapping analysis of the social work field, which shows its conceptual structure and scientific evolution. Methods: Science Mapping Analysis Software Tool, a bibliometric science mapping tool based on co-word analysis and h-index, is applied using a sample of 18,794 research articles published from 1930 to 2012 in 25 main social work journals indexed in the Journal Citation Reports of the Web of Science. Results: Published research social work field concentrated in eight main thematic areas: children, social services, health care, violence, women, HIV/AIDS, social workers, and education. HIV/AIDS and violence have recently attracted the interest of the social word scientific community, while the rest are classical thematic areas that still attract the interest and efforts of the researchers. Conclusion: This conceptual and empirical analysis shows how research themes have evolved in social work."
2117331411,Using the h‐index to rank influential British researchers in information science and librarianship,"Abstract The recently developed h ‐index has been applied to the literature produced by senior British‐based academics in librarianship and information science. The majority of those evaluated currently hold senior positions in UK information science and librarianship departments; however, a small number of staff in other departments and retired “founding fathers” were analyzed as well. The analysis was carried out using the Web of Science (Thomson Scientific, Philadelphia, PA) for the years from 1992 to October 2005, and included both second‐authored papers and self‐citations. The top‐ranking British information scientist, Peter Willett, has an h ‐index of 31. However, it was found that Eugene Garfield, the founder of modern citation studies, has an even higher h ‐index of 36. These results support other studies suggesting that the h ‐index is a useful tool in the armory of bibliometrics."
2065873852,Editorial Peer Reviewers' Recommendations at a General Medical Journal: Are They Reliable and Do Editors Care?,"Background Editorial peer review is universally used but little studied. We examined the relationship between external reviewers' recommendations and the editorial outcome of manuscripts undergoing external peer-review at the Journal of General Internal Medicine (JGIM). Methodology/Principal Findings We examined reviewer recommendations and editors' decisions at JGIM between 2004 and 2008. For manuscripts undergoing peer review, we calculated chance-corrected agreement among reviewers on recommendations to reject versus accept or revise. Using mixed effects logistic regression models, we estimated intra-class correlation coefficients (ICC) at the reviewer and manuscript level. Finally, we examined the probability of rejection in relation to reviewer agreement and disagreement. The 2264 manuscripts sent for external review during the study period received 5881 reviews provided by 2916 reviewers; 28% of reviews recommended rejection. Chance corrected agreement (kappa statistic) on rejection among reviewers was 0.11 (p<.01). In mixed effects models adjusting for study year and manuscript type, the reviewer-level ICC was 0.23 (95% confidence interval [CI], 0.19–0.29) and the manuscript-level ICC was 0.17 (95% CI, 0.12–0.22). The editors' overall rejection rate was 48%: 88% when all reviewers for a manuscript agreed on rejection (7% of manuscripts) and 20% when all reviewers agreed that the manuscript should not be rejected (48% of manuscripts) (p<0.01). Conclusions/Significance Reviewers at JGIM agreed on recommendations to reject vs. accept/revise at levels barely beyond chance, yet editors placed considerable weight on reviewers' recommendations. Efforts are needed to improve the reliability of the peer-review process while helping editors understand the limitations of reviewers' recommendations."
2153722135,Scientific Journal Publishing: Yearly Volume and Open Access Availability.,"Introduction. We estimate the total yearly volume of peer-reviewed scientific journal articles published world-wide as well as the share of these articles available openly on the Web either directly or as copies in e-print repositories.Method. We rely on data from two commercial databases (ISI and Ulrich's Periodicals Directory) supplemented by sampling and Google searches.Analysis. A central issue is the finding that ISI-indexed journals publish far more articles per year (111) than non ISI-indexed journals (26), which means that the total figure we obtain is much lower than many earlier estimates. Our method of analysing the number of repository copies (green open access) differs from several earlier studies which have studied the number of copies in identified repositories, since we start from a random sample of articles and then test if copies can be found by a Web search engine.Results. We estimate that in 2006 the total number of articles published was approximately 1,350,000. Of this number 4.6% became immediately openly available and an additional 3.5% after an embargo period of, typically, one year. Furthermore, usable copies of 11.3% could be found in subject-specific or institutional repositories or on the home pages of the authors.Conclusions. We believe our results are the most reliable so far published and, therefore, should be useful in the on-going debate about Open Access among both academics and science policy makers. The method is replicable and also lends itself to longitudinal studies in the future."
2118215647,Mendeley readership counts: An investigation of temporal and disciplinary differences,"Scientists and managers using citation‐based indicators to help evaluate research cannot evaluate recent articles because of the time needed for citations to accrue. Reading occurs before citing, however, and so it makes sense to count readers rather than citations for recent publications. To assess this, M endeley readers and citations were obtained for articles from 2004 to late 2014 in five broad categories (agriculture, business, decision science, pharmacy, and the social sciences) and 50 subcategories. In these areas, citation counts tended to increase with every extra year since publication, and readership counts tended to increase faster initially but then stabilize after about 5 years. The correlation between citations and readers was also higher for longer time periods, stabilizing after about 5 years. Although there were substantial differences between broad fields and smaller differences between subfields, the results confirm the value of M endeley reader counts as early scientific impact indicators."
1666134154,The w‐index: A measure to assess scientific impact by focusing on widely cited papers,"I propose a new measure, the w-index, as a particularly simple and useful way to assess the integrated impact of a researcher's work, especially his or her excellent papers. The w-index can be defined as follows: If w of a researcher's papers have at least 10w citations each and the other papers have fewer than 10(w+1) citations, his/her w-index is w. It is a significant improvement of the h-index."
2082603871,Mapping knowledge domains,"The term “mapping knowledge domains” was chosen to describe a newly evolving interdisciplinary area of science aimed at the process of charting, mining, analyzing, sorting, enabling navigation of, and displaying knowledge. This field is aimed at easing information access, making evident the structure of knowledge, and allowing seekers of knowledge to succeed in their endeavors. Although thousands of years old, this area has undergone a sea change in the last 15 years, a change fostered by an explosion of the amount of information available, the accessibility of that information due to electronic storage, and the new techniques of analysis, retrieval, and visualization that are made possible by vast increases in computational storage capacity and processing speed and power. Many of us are so involved in the new ways of accessing knowledge that we have forgotten how recent is the change to computerized knowledge retrieval with search engines operating on the World Wide Web. Remarkable as these changes are to date, they are only a hint of the transformation to come. The Arthur M. Sackler Colloquium on Mapping Knowledge Domains, held at the Arnold and Mabel Beckman Center of the National Academies of Sciences and Engineering in Irvine, CA, May 9-11, 2003, was designed to showcase the ongoing developments in this transformation and provide pointers toward the directions it will move.The changes that are taking place profoundly affect the way we access and use information. Scientists, academics, and librarians have historically worked hard to codify, classify, and organize knowledge, thereby making it useful and accessible. The day is fast approaching when all this knowledge will be coded electronically, but mixed in a vast and largely disorganized and often unreliable sea of mostly recent information. Fishing this sea for desired information is presently no easy task and will continue to increase …"
2163187547,Scopus database: a review,"Abstract The Scopus database provides access to STM journal articles and the references included in those articles, allowing the searcher to search both forward and backward in time. The database can be used for collection development as well as for research. This review provides information on the key points of the database and compares it to Web of Science. Neither database is inclusive, but complements each other. If a library can only afford one, choice must be based in institutional needs."
2398137172,State and Analysis of Scientific Journals in the Field of “Economic Sciences” for the Period 1995-2014,"Abstract SCI-E and SSCI are one of the main citation databases (CDB) in the world in the field of natural and applied (SCI-E) and social (SSCI) scientific disciplines. In 2014, within the SCIE all scientific fields of natural and applied science disciplines are classified into 176 categories, and within SSCI all scientific fields of social science disciplines are classified into 56 categories. Scientific journals in the field of “Economic Sciences” are sorted within the SSCI in 8 (eight) categories, plus one category within the SCI-E. The aim of the paper is to show the state and analysis of scientific journals indexed in SSCI and SCI-E citation databases (CDB) for 9 (nine) listed categories in the field of “Economic Sciences” for the period 1995-2014. The number of different scientific journals for 9 (nine) listed categories in the field of “Economic Sciences” for a period of 1997-2014, increased for 455 journals (from 415 journals in 1995 to 860 journals in 2014) with cumulative growth index (CGI) of 207,23% compared to 1997. Best ranked category among the nine listed categories in the field of “Economic Sciences” was the category “Economics” (Econ), which according to the number of journals for the entire period 1995-2014 was always ranked first (top-one) within SSCI. The largest increase in the number of scientific journals for the period 1995-2014 for all nine listed categories in the field of “Economic Sciences” had the category “Management” (Manag)."
2915612428,Predicting the citations of scholarly paper,"Citation prediction of scholarly papers is of great significance in guiding funding allocations, recruitment decisions, and rewards. However, little is known about how citation patterns evolve over time. By exploring the inherent involution property in scholarly paper citation, we introduce the Paper Potential Index (PPI) model based on four factors: inherent quality of scholarly paper, scholarly paper impact decaying over time, early citations, and early citers' impact. In addition, by analyzing factors that drive citation growth, we propose a multi-feature model for impact prediction. Experimental results demonstrate that the two models improve the accuracy in predicting scholarly paper citations. Compared to the multi-feature model, the PPI model yields superior predictive performance in terms of range-normalized RMSE. The PPI model better interprets the changes in citation, without the need to adjust parameters. Compared to the PPI model, the multi-feature model performs better prediction in terms of Mean Absolute Percentage Error and Accuracy; however, their predictive performance is more dependent on the parameter adjustment."
1498995181,"A New Era in Citation and Bibliometric Analyses: Web of Science, Scopus, and Google Scholar","Academic institutions, federal agencies, publishers, editors, authors, and librarians increasingly rely on citation analysis for making hiring, promotion, tenure, funding, and/or reviewer and journal evaluation and selection decisions. The Institute for Scientific Information's (ISI) citation databases have been used for decades as a starting point and often as the only tools for locating citations and/or conducting citation analyses. ISI databases (or Web of Science), however, may no longer be adequate as the only or even the main sources of citations because new databases and tools that allow citation searching are now available. Whether these new databases and tools complement or represent alternatives to Web of Science (WoS) is important to explore. Using a group of 15 library and information science faculty members as a case study, this paper examines the effects of using Scopus and Google Scholar (GS) on the citation counts and rankings of scholars as measured by WoS. The paper discusses the strengths and weaknesses of WoS, Scopus, and GS, their overlap and uniqueness, quality and language of the citations, and the implications of the findings for citation analysis. The project involved citation searching for approximately 1,100 scholarly works published by the study group and over 200 works by a test group (an additional 10 faculty members). Overall, more than 10,000 citing and purportedly citing documents were examined. WoS data took about 100 hours of collecting and processing time, Scopus consumed 200 hours, and GS a grueling 3,000 hours."
2785598616,Field normalization of scientometric indicators,"When scientometric indicators are used to compare research units active in different scientific fields, there often is a need to make corrections for differences between fields, for instance differences in publication, collaboration, and citation practices. Field-normalized indicators aim to make such corrections. The design of these indicators is a significant challenge. We discuss the main issues in the design of field-normalized indicators, and we present an overview of different approaches that have been developed for dealing with the problem of field normalization. We also discuss how field-normalized indicators can be evaluated, and we consider the sensitivity of scientometric analyses to the choice of a field normalization approach."
2034083929,Looking for Landmarks: The Role of Expert Review and Bibliometric Analysis in Evaluating Scientific Publication Outputs,"To compare expert assessment with bibliometric indicators as tools to assess the quality and importance of scientific research papers.Shortly after their publication in 2005, the quality and importance of a cohort of nearly 700 Wellcome Trust (WT) associated research papers were assessed by expert reviewers; each paper was reviewed by two WT expert reviewers. After 3 years, we compared this initial assessment with other measures of paper impact.Shortly after publication, 62 (9%) of the 687 research papers were determined to describe at least a 'major addition to knowledge' -6 were thought to be 'landmark' papers. At an aggregate level, after 3 years, there was a strong positive association between expert assessment and impact as measured by number of citations and F1000 rating. However, there were some important exceptions indicating that bibliometric measures may not be sufficient in isolation as measures of research quality and importance, and especially not for assessing single papers or small groups of research publications.When attempting to assess the quality and importance of research papers, we found that sole reliance on bibliometric indicators would have led us to miss papers containing important results as judged by expert review. In particular, some papers that were highly rated by experts were not highly cited during the first three years after publication. Tools that link expert peer reviews of research paper quality and importance to more quantitative indicators, such as citation analysis would be valuable additions to the field of research assessment and evaluation."
1572894270,An analysis of publication productivity for 1225 academic neurosurgeons and 99 departments in the United States,"Object Bibliometrics is defined as the study of statistical and mathematical methods used to quantitatively analyze scientific literature. The application of bibliometrics in neurosurgery is in its infancy. The authors calculate a number of publication productivity measures for almost all academic neurosurgeons and departments within the US. Methods The h-index, g-index, m-quotient, and contemporary h-index (h c -index) were calculated for 1225 academic neurosurgeons in 99 (of 101) programs listed by the Accreditation Council for Graduate Medical Education in January 2013. Three currently available citation databases were used: Google Scholar, Scopus, and Web of Science. Bibliometric profiles were created for each surgeon. Comparisons based on academic rank (that is, chairperson, professor, associate, assistant, and instructor), sex, and subspecialties were performed. Departments were ranked based on the summation of individual faculty h-indices. Calculations were carried out from January to February 2013. Results The median h-index, g-index, h c -index, and m-quotient were 11, 20, 8, and 0.62, respectively. All indices demonstrated a positive relationship with increasing academic rank (p < 0.001). The median h-index was 11 for males (n = 1144) and 8 for females (n = 81). The h-index, g-index and h c -index significantly varied by sex (p < 0.001). However, when corrected for academic rank, this difference was no longer significant. There was no difference in the m-quotient by sex. Neurosurgeons with subspecialties in functional/epilepsy, peripheral nerve, radiosurgery, neuro-oncology/skull base, and vascular have the highest median h-indices; general, pediatric, and spine neurosurgeons have the lowest median h-indices. By summing the manually calculated Scopus h-indices of all individuals within a department, the top 5 programs for publication productivity are University of California, San Francisco; Barrow Neurological Institute; Johns Hopkins University; University of Pittsburgh; and University of California, Los Angeles. Conclusions This study represents the most detailed publication analysis of academic neurosurgeons and their programs to date. The results for the metrics presented should be viewed as benchmarks for comparison purposes. It is our hope that organized neurosurgery will adopt and continue to refine bibliometric profiling of individuals and departments."
1784280640,The Assessment of Research Quality in UK Universities: Peer Review or Metrics?,"This paper investigates the extent to which the outcomes of the 2008 Research Assessment Exercise in the UK, determined by peer review, can be explained by a set of quantitative indicators. Three cognate units of assessment are examined in detail: business and management, economics and econometrics, and accounting and finance. The main finding is that each of the three components of research activity (namely, research output, esteem and research environment) is highly correlated with various quantitative indicators. A further finding is that the judgement of the Research Assessment Exercise panels was biased in favour of Russell Group universities. There is also evidence of bias by the economics and econometrics panel. The results support the use of quantitative indicators in the research assessment process, particularly a journal quality index. Requiring the panels to take bibliometric indicators into account should help not only to reduce the workload of panels but also to mitigate the problem of implicit bias."
3121567788,IDENTIFYING THE EFFECT OF OPEN ACCESS ON CITATIONS USING A PANEL OF SCIENCE JOURNALS,"An open‐access journal allows free online access to its articles, obtaining revenue from fees charged to submitting authors or from institutional support. Using panel data on science journals, we are able to circumvent problems plaguing previous studies of the impact of open access on citations. In contrast to the huge effects found in these previous studies, we find a more modest effect: moving from paid to open access increases cites by 8% on average in our sample. The benefit is concentrated among top‐ranked journals. In fact, open access causes a statistically significant reduction in cites to the bottom‐ranked journals in our sample, leading us to conjecture that open access may intensify competition among articles for readers' attention, generating losers as well as winners . ( JEL L17, O33)"
2790320647,"Authorship, citations, acknowledgments and visibility in social media: Symbolic capital in the multifaceted reward system of science","The reward system of science is undergoing significant changes, as traditional indicators compete with initiatives that offer novel means of disseminating and assessing scholarly impact. This article considers a number of aspects of this reward system, including authorship, citations, acknowledgements and the growing use of social media platforms by academics, with an eye towards identifying contemporary issues relating to scholarly communication practices, as understood through the perspectives of Bourdieu’s symbolic capital and Merton’s recognition framework. The article posits that, while scientific capital remains the foundation upon which the reward system of science is built, this system is revealing itself to be more and more multifaceted, extremely complex, and facing increasing tension between its traditional means of evaluation and the potential of new indicators in the digital era. The article presents an extended literature review, as well as recommendations for further consideration and empirical research. A better understanding of the perceptions of academics would be necessary to properly assess the effects of these new indicators on scholarly communication practices and the reward system of science."
1988169643,Altmetrics: nuevos indicadores para la comunicación científica en la Web 2.0,"En el presente trabajo se realiza una revision de las altmetrics o indicadores alternativos. Este concepto se define como la creacion y estudio de nuevos indicadores, basados en la web 2.0, para el analisis de la actividad cientifica y academica. La idea que subyace es que, por ejemplo, las menciones en blogs, el numero de tuits o el de personas que guardan un articulo en su gestor de referencias puede ser una medida valida del uso y repercusion de las publicaciones cientificas. En este sentido, estas medidas se han situado en el centro del debate de los estudios bibliometricos cobrando especial relevancia. En el articulo se ilustran en primer lugar las plataformas e indicadores principales de este tipo de medidas, para posteriormente estudiar un conjunto de trabajos del ambito de la comunicacion, comparando el numero de citas recibidas con sus indicadores 2.0. Los resultados senalan que los articulos mas citados de la disciplina en los ultimos anos tambien presentan indicadores significativamente mas elevados de altmetrics. Seguidamente se realiza un repaso por los principales estudios empiricos realizados, deteniendonos en las correlaciones entre indicadores bibliometricos y alternativos. Se finaliza, a modo de reflexion, senalando las principales limitaciones y el papel que las altmetrics pueden desempenar a la hora de captar la repercusion de la investigacion en las plataformas de la web 2.0."
1996299733,"International coauthorship and citation impact: A bibliometric study of six LIS journals, 1980-2008","International collaborative papers are increasingly common in journals of many disciplines. These types of papers are often cited more frequently. To identify the coauthorship trends within Library and Information Science (LIS), this study analyzed 7,489 papers published in six leading publications (ARIST, IP&M, JAMIA, JASIST, MISQ, and Scientometrics) over the last three decades. Logistic regression tested the relationships between citations received and seven factors: authorship type, author's subregion, country income level, publication year, number of authors, document type, and journal title. The main authorship type since 1995 was national collaboration. It was also the dominant type for all publications studied except ARIST, and for all regions except Africa. For citation counts, the logistic regression analysis found all seven factors were significant. Papers that included international collaboration, Northern European authors, and authors in high-income nations had higher odds of being cited more. Papers from East Asia, Southeast Asia, and Southern Europe had lower odds than North American papers. As discussed in the bibliometric literature, Merton's Matthew Effect sheds light on the differential citation counts based on the authors' subregion. This researcher proposes geographies of invisible colleagues and a geographic scope effect to further investigate the relationships between author geographic affiliation and citation impact."
1505404142,The thematic orientation of publications mentioned on social media,"Purpose The purpose of this paper is to analyze the disciplinary orientation of scientific publications that were mentioned on different social media platforms, focussing on their differences and similarities with citation counts."
1522052181,The Open Access citation advantage: Studies and results to date,"This paper presents a summary of reported studies on the Open Access citation advantage. There is a brief introduction to the main issues involved in carrying out such studies, both methodological and interpretive. The study listing provides some details of the coverage, methodological approach and main conclusions of each study."
2910210452,The effect of publishing peer review reports on referee behavior in five scholarly journals,"Abstract To increase transparency in science, some scholarly journals are publishing peer review reports. But it is unclear how this practice affects the peer review process. Here, we examine the effect of publishing peer review reports on referee behavior in five scholarly journals involved in a pilot study at Elsevier. By considering 9,220 submissions and 18,525 reviews from 2010 to 2017, we measured changes both before and during the pilot and found that publishing reports did not significantly compromise referees’ willingness to review, recommendations, or turn-around times. Younger and non-academic scholars were more willing to accept to review and provided more positive and objective recommendations. Male referees tended to write more constructive reports during the pilot. Only 8.1% of referees agreed to reveal their identity in the published report. These findings suggest that open peer review does not compromise the process, at least when referees are able to protect their anonymity."
2343014812,Who's downloading pirated papers? Everyone,Data from the controversial website Sci-Hub reveal that the whole world turns to it for journal articles.
874287882,The effect of open access and downloads ('hits') on citation impact: a bibliography of studies,"Since 1998 studies have shown that open access increases impact as measured by the number of citations. This chronological and comprehensive bibliography of those reports provides a way of investigating the meaning of that statement, to understand its effect, to recognise the significant factors involved, and identify the causal relationships between the respective factors. Reviews in 2010 suggest a ratio of between 5.5:1 and 7:1 of studies showing an open access citation advantage against those that do not. The bibliography also lists the Web tools available to measure impact. It is a focused bibliography, on the relationship between impact and access. It does not attempt to cover citation impact, or other related topics such as open access, more generally, although some key papers in these areas are listed as jump-off points for wider study. Other important effects can be found in the area of research assessment. There is an occasionally updated section on the correlation between research access, impact and assessment. This is a snapshot of the bibliography taken towards the end June 2013. The research on this topic continues, as does the updated version of the bibliography."
4292405002,Publish or impoverish,"Purpose: The purpose of this study is to present the landscape of the cash-per-publication reward policy in China and reveal its trend since the late 1990s. Design/methodology/approach: This study is based on the analysis of 168 university documents regarding the cash-per-publication reward policy at 100 Chinese universities. Findings: Chinese universities offer cash rewards from 30 to 165,000 USD for papers published in journals indexed by Web of Science (WoS), and the average reward amount has been increasing for the past 10 years. Originality/value: The cash-per-publication reward policy in China has never been systematically studied and investigated before except for in some case studies. This is the first paper that reveals the landscape of the cash-per-publication reward policy in China."
2913304944,Blacklists and Whitelists To Tackle Predatory Publishing: a Cross-Sectional Comparison and Thematic Analysis,"We aimed to develop an in-depth understanding of quality criteria for scholarly journals by analyzing journals and publishers indexed in blacklists of predatory journals and whitelists of legitimate journals and the lists' inclusion criteria. To quantify content overlaps between blacklists and whitelists, we employed the Jaro-Winkler string metric. To identify topics addressed by the lists' inclusion criteria and to derive their concepts, we conducted qualitative coding. We included two blacklists (Beall's and Cabells Scholarly Analytics') and two whitelists (the Directory of Open Access Journals' and Cabells Scholarly Analytics'). The number of journals per list ranged from 1,404 to 12,357, and the number of publishers ranged from 473 to 5,638. Seventy-two journals and 42 publishers were included in both a blacklist and a whitelist. Seven themes were identified in the inclusion criteria: (i) peer review; (ii) editorial services; (iii) policy; (iv) business practices; (v) publishing, archiving, and access; (vi) website; and (vii) indexing and metrics. Business practices accounted for almost half of the blacklists' criteria, whereas whitelists gave more emphasis to criteria related to policy. Criteria could be allocated to four concepts: (i) transparency, (ii) ethics, (iii) professional standards, and (iv) peer review and other services. Whitelists gave most weight to transparency. Blacklists focused on ethics and professional standards. Whitelist criteria were easier to verify than those used in blacklists. Both types gave little emphasis to quality of peer review. Overall, the results show that there is overlap of journals and publishers between blacklists and whitelists. Lists differ in their criteria for quality and the weight given to different dimensions of quality. Aspects that are central but difficult to verify receive little attention.IMPORTANCE Predatory journals are spurious scientific outlets that charge fees for editorial and publishing services that they do not provide. Their lack of quality assurance of published articles increases the risk that unreliable research is published and thus jeopardizes the integrity and credibility of research as a whole. There is increasing awareness of the risks associated with predatory publishing, but efforts to address this situation are hampered by the lack of a clear definition of predatory outlets. Blacklists of predatory journals and whitelists of legitimate journals have been developed but not comprehensively examined. By systematically analyzing these lists, this study provides insights into their utility and delineates the different notions of quality and legitimacy in scholarly publishing used. This study contributes to a better understanding of the relevant concepts and provides a starting point for the development of a robust definition of predatory journals."
4241081299,Can scientific journals be classified in terms of aggregated journal‐journal citation relations using the Journal Citation Reports?,"Abstract The aggregated citation relations among journals included in the Science Citation Index provide us with a huge matrix, which can be analyzed in various ways. By using principal component analysis or factor analysis, the factor scores can be employed as indicators of the position of the cited journals in the citing dimensions of the database. Unrotated factor scores are exact, and the extraction of principal components can be made stepwise because the principal components are independent. Rotation may be needed for the designation, but in the rotated solution a model is assumed. This assumption can be legitimated on pragmatic or theoretical grounds. Because the resulting outcomes remain sensitive to the assumptions in the model, an unambiguous classification is no longer possible in this case. However, the factor‐analytic solutions allow us to test classifications against the structures contained in the database; in this article the process will be demonstrated for the delineation of a set of biochemistry journals."
1864634822,Emerging trends in peer review—a survey,"“Classical peer review” has been subject to intense criticism for slowing down the publication process, bias against specific categories of paper and author, unreliability, inability to detect errors and fraud, unethical practices, and the lack of recognition for unpaid reviewers. This paper surveys innovative forms of peer review that attempt to address these issues. Based on an initial literature review, we construct a sample of 81 channels of scientific communication covering all forms of review identified by the survey and analyze the review mechanisms used by each channel. We identify two major trends: the rapidly expanding role of pre-print servers (e.g. ArXiv) that dispense with traditional peer review altogether and the growth of “non-selective review”, focusing on papers’ scientific quality rather than their perceived importance and novelty. Other potentially important developments include forms of “open review”, which remove reviewer anonymity and interactive review, as well as new mechanisms for post-publication review and out-of-channel reader commentary, especially critical commentary targeting high profile papers. One of the strongest findings of the survey is the persistence of major differences between the peer review processes used by different disciplines. None of these differences is likely to disappear in the foreseeable future. The most likely scenario for the coming years is thus continued diversification, in which different review mechanisms serve different author and reader needs. Relatively little is known about the impact of these innovations on the problems they address. These are important questions for future quantitative research."
3104402874,Ranking journals: could Google Scholar Metrics be an alternative to Journal Citation Reports and Scimago Journal Rank?,"The launch of Google Scholar Metrics as a tool for assessing scientific journals may be serious competition for Thomson Reuters Journal Citation Reports, and for Scopus powered Scimago Journal Rank. A review of these bibliometric journal evaluation products is performed. We compare their main characteristics from different approaches: coverage, indexing policies, search and visualization, bibliometric indicators, results analysis options, economic cost and differences in their ranking of journals. Despite its shortcomings, Google Scholar Metrics is a helpful tool for authors and editors in identifying core journals. As an increasingly useful tool for ranking scientific journals, it may also challenge established journals products"
2171827218,Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial,"To see whether telling peer reviewers that their signed reviews of original research papers might be posted on the BMJ's website would affect the quality of their reviews.Randomised controlled trial.A large international general medical journal based in the United Kingdom.541 authors, 471 peer reviewers, and 12 editors.Consecutive eligible papers were randomised either to have the reviewer's signed report made available on the BMJ's website alongside the published paper (intervention group) or to have the report made available only to the author-the BMJ's normal procedure (control group). The intervention was the act of revealing to reviewers-after they had agreed to review but before they undertook their review-that their signed report might appear on the website.The main outcome measure was the quality of the reviews, as independently rated on a scale of 1 to 5 using a validated instrument by two editors and the corresponding author. Authors and editors were blind to the intervention group. Authors rated review quality before the fate of their paper had been decided. Additional outcomes were the time taken to complete the review and the reviewer's recommendation regarding publication.558 manuscripts were randomised, and 471 manuscripts remained after exclusions. Of the 1039 reviewers approached to take part in the study, 568 (55%) declined. Two editors' evaluations of the quality of the peer review were obtained for all 471 manuscripts, with the corresponding author's evaluation obtained for 453. There was no significant difference in review quality between the intervention and control groups (mean difference for editors 0.04, 95% CI -0.09 to 0.17; for authors 0.06, 95% CI -0.09 to 0.20). Any possible difference in favour of the control group was well below the level regarded as editorially significant. Reviewers in the intervention group took significantly longer to review (mean difference 25 minutes, 95% CI 3.0 to 47.0 minutes).Telling peer reviewers that their signed reviews might be available in the public domain on the BMJ's website had no important effect on review quality. Although the possibility of posting reviews online was associated with a high refusal rate among potential peer reviewers and an increase in the amount of time taken to write a review, we believe that the ethical arguments in favour of open peer review more than outweigh these disadvantages."
1984343572,"Interdisciplinary research: meaning, metrics and nurture","Journal Article Interdisciplinary research: meaning, metrics and nurture Get access Alan L Porter, Alan L Porter Search for other works by this author on: Oxford Academic Google Scholar J David Roessner, J David Roessner Search for other works by this author on: Oxford Academic Google Scholar Alex S Cohen, Alex S Cohen Search for other works by this author on: Oxford Academic Google Scholar Marty Perreault Marty Perreault Search for other works by this author on: Oxford Academic Google Scholar Research Evaluation, Volume 15, Issue 3, December 2006, Pages 187–195, https://doi.org/10.3152/147154406781775841 Published: 01 December 2006"
3039653926,Communication in international collaborative research teams. A review of the state of the art and open research questions,"In Communication Science, international scholarly communication and collaboration practice still remain unknown territory. Therefore, a systematic review of the state of the art on scholarly communication practice in international research collaborations (IRCs) was carried out that included a broad spectrum of disciplines and research fields such as Communication Science, Business and Management Studies, Sociology, science studies, and the science of team science. A sample of 168 contributions focusing on IRCs were identified. The paper outlines focus and methodological designs of those contributions, provides insights into the composition of the observed IRCs, summarizes the perspectives of the disciplines and research fields, presents the insights into communication structures and processes in IRCs, and discusses the aspect of team diversity, which some studies indicate as relevant for communication practice in IRCs. Overall, research largely focuses on the structural dimension of communication, while empirical analyses on the actual communication processes among scientists in IRCs are still rare. Secondly, research is missing on how team complexity is dealt with in IRCs and what impact it has on collaboration processes and success. A third and fourth research gap are identified regarding the use of a joint collaboration language and the communication processes in Social Sciences and Humanities. Future research should broaden its analytical focus to fill those gaps. This would provide important insights from an epistemological and a practical perspective, by offering the foundation for the development of guidelines and toolkits for future IRCs, thus contributing to the success of such forms of research and knowledge creation."
3014226787,A review of the literature on ethical issues related to scientific authorship,"The article at hand presents the results of a literature review on the ethical issues related to scientific authorship. These issues are understood as questions and/or concerns about obligations, values or virtues in relation to reporting, authorship and publication of research results. For this purpose, the Web of Science core collection was searched for English resources published between 1945 and 2018, and a total of 324 items were analyzed. Based on the review of the documents, ten ethical themes have been identified, some of which entail several ethical issues. Ranked on the basis of their frequency of occurrence these themes are: 1) attribution, 2) violations of the norms of authorship, 3) bias, 4) responsibility and accountability, 5) authorship order, 6) citations and referencing, 7) definition of authorship, 8) publication strategy, 9) originality, and 10) sanctions. In mapping these themes, the current article explores major ethical issue and provides a critical discussion about the application of codes of conduct, various understandings of culture, and contributing factors to unethical behavior."
1889778676,Open access to scientific publications - an analysis of the barriers to change,"One of the effects of the Internet is that the dissemination of scientific publications in a few years has migrated to electronic formats. The basic business practices between libraries and publishers for selling and buying the content, however, have not changed much. In protest against the high subscription prices of mainstream publishers, scientists have started Open Access (OA) journals and e-print repositories, which distribute scientific information freely. Despite widespread agreement among academics that OA would be the optimal distribution mode for publicly financed research results, such channels still constitute only a marginal phenomenon in the global scholarly communication system. This paper discusses, in view of the experiences of the last ten years, the many barriers hindering a rapid proliferation of Open Access. The discussion is structured according to the main OA channels; peer-reviewed journals for primary publishing, subjectspecific and institutional repositories for secondary parallel publishing. It also discusses the types of barriers, which can be classified as consisting of the legal framework, the information technology infrastructure, business models, indexing services and standards, the academic reward system, marketing, and critical mass."
2104613635,Interactive overlays of journals and the measurement of interdisciplinarity on the basis of aggregated journal–journal citations,"Using the option Analyze Results with the Web of Science, one can directly generate overlays onto global journal maps of science. The maps are based on the 10,000+ journals contained in the Journal Citation Reports ( JCR ) of the Science and Social Sciences Citation Indices (2011). The disciplinary diversity of the retrieval is measured in terms of Rao‐Stirling's “quadratic entropy” (Izsák & Papp, 1995). Since this indicator of interdisciplinarity is normalized between 0 and 1, interdisciplinarity can be compared among document sets and across years, cited or citing. The colors used for the overlays are based on Blondel, Guillaume, Lambiotte, and Lefebvre's (2008) community‐finding algorithms operating on the relations among journals included in the JCR . The results can be exported from VOSViewer with different options such as proportional labels, heat maps, or cluster density maps. The maps can also be web‐started or animated (e.g., using PowerPoint). The “citing” dimension of the aggregated journal–journal citation matrix was found to provide a more comprehensive description than the matrix based on the cited archive. The relations between local and global maps and their different functions in studying the sciences in terms of journal literatures are further discussed: Local and global maps are based on different assumptions and can be expected to serve different purposes for the explanation."
3102480278,How is open access accused of being predatory? The impact of Beall's lists of predatory journals on academic publishing,"The aim of this paper is to investigate how predatory journals are characterized by authors who write about such journals. We emphasize the ways in which predatory journals have been conflated with—or distinguished from—open access journals. We created a list of relevant publications on predatory publishing using four databases: Web of Science, Scopus, Dimensions, and Microsoft Academic. We included 280 English-language publications in the review according to their contributions to the discussions on predatory publishing. Then, we coded and qualitatively analyzed these publications. The findings show the profound influence of Jeffrey Beall, who composed and maintained himself lists of predatory publishers and journals, on the whole discussion on predatory publishing. The major themes by which Beall has characterized predatory journals are widely present in non-Beall publications. Moreover, 122 papers we reviewed combined predatory publishing with open access using similar strategies as Beall. The overgeneralization of the flaws of some open access journals to the entire open access movement has led to unjustified prejudices among the academic community toward open access. This is the first large-scale study that systematically examines how predatory publishing is defined in the literature."
4242734663,Mathematical theory of the h‐ and g‐index in case of fractional counting of authorship,"Abstract This article studies the h‐index (Hirsch index) and the g‐index of authors, in case one counts authorship of the cited articles in a fractional way. There are two ways to do this: One counts the citations to these papers in a fractional way or one counts the ranks of the papers in a fractional way as credit for an author. In both cases, we define the fractional h‐ and g‐indexes, and we present inequalities (both upper and lower bounds) between these fractional h‐ and g‐indexes and their corresponding unweighted values (also involving, of course, the coauthorship distribution). Wherever applicable, examples and counterexamples are provided. In a concrete example (the publication citation list of the present author), we make explicit calculations of these fractional h‐ and g‐indexes and show that they are not very different from the unweighted ones."
2120867647,Predictive effects of structural variation on citation counts,"Abstract A critical part of a scientific activity is to discern how a new idea is related to what we know and what may become possible. As the number of new scientific publications arrives at a rate that rapidly outpaces our capacity of reading, analyzing, and synthesizing scientific knowledge, we need to augment ourselves with information that can effectively guide us through the rapidly growing intellectual space. In this article, we address a fundamental issue concerning what kinds of information may serve as early signs of potentially valuable ideas. In particular, we are interested in information that is routinely available and derivable upon the publication of a scientific paper without assuming the availability of additional information such as its usage and citations. We propose a theoretical and computational model that predicts the potential of a scientific publication in terms of the degree to which it alters the intellectual structure of the state of the art. The structural variation approach focuses on the novel boundary‐spanning connections introduced by a new article to the intellectual space. We validate the role of boundary‐spanning in predicting future citations using three metrics of structural variation—namely, modularity change rate, cluster linkage, and Centrality Divergence—along with more commonly studied predictors of citations such as the number of coauthors, the number of cited references, and the number of pages. Main effects of these factors are estimated for five cases using zero‐inflated negative binomial regression models of citation counts. Key findings indicate that (a) structural variations measured by cluster linkage are a better predictor of citation counts than are the more commonly studied variables such as the number of references cited, (b) the number of coauthors and the number of references are both good predictors of global citation counts to a lesser extent, and (c) the Centrality Divergence metric is potentially valuable for detecting boundary‐spanning activities at interdisciplinary levels. The structural variation approach offers a new way to monitor and discern the potential of newly published papers in context. The boundary‐spanning mechanism offers a conceptually simplified and unifying explanation of the roles played by commonly studied extrinsic properties of a publication in the study of citation behavior."
2016553928,Does the Committee Peer Review Select the Best Applicants for Funding? An Investigation of the Selection Process for Two European Molecular Biology Organization Programmes,"Does peer review fulfill its declared objective of identifying the best science and the best scientists? In order to answer this question we analyzed the Long-Term Fellowship and the Young Investigator programmes of the European Molecular Biology Organization. Both programmes aim to identify and support the best post doctoral fellows and young group leaders in the life sciences. We checked the association between the selection decisions and the scientific performance of the applicants. Our study involved publication and citation data for 668 applicants to the Long-Term Fellowship programme from the year 1998 (130 approved, 538 rejected) and 297 applicants to the Young Investigator programme (39 approved and 258 rejected applicants) from the years 2001 and 2002. If quantity and impact of research publications are used as a criterion for scientific achievement, the results of (zero-truncated) negative binomial models show that the peer review process indeed selects scientists who perform on a higher level than the rejected ones subsequent to application. We determined the extent of errors due to over-estimation (type I errors) and under-estimation (type 2 errors) of future scientific performance. Our statistical analyses point out that between 26% and 48% of the decisions made to award or reject an application show one of both error types. Even though for a part of the applicants, the selection committee did not correctly estimate the applicant's future performance, the results show a statistically significant association between selection decisions and the applicants' scientific achievements, if quantity and impact of research publications are used as a criterion for scientific achievement."
2120016957,Do age and professional rank influence the order of authorship in scientific publications? Some evidence from a micro-level perspective,"Scientific authorship has important implications in science since it reflects the contribution to research of the different individual scientists and it is considered by evaluation committees in research assessment processes. This study analyses the order of authorship in the scientific output of 1,064 permanent scientists at the Spanish CSIC (WoS, 1994-2004). The influence of age, professional rank and bibliometric profile of scientists over the position of their names in the byline of publications is explored in three different research areas: Biology and Biomedicine, Materials Science and Natural Resources. There is a strong trend for signatures of younger researchers and those in the lower professional ranks to appear in the first position (junior signing pattern), while more veteran or highly-ranked ones, who tend to play supervisory functions in research, are proportionally more likely to sign in the last position (senior signing pattern). Professional rank and age have an effect on authorship order in the three fields analysed, but there are inter-field differences. Authorship patterns are especially marked in the most collaboration-intensive field (i.e. Biology and Biomedicine), where professional rank seems to be more significant than age in determining the role of scientists in research as seen through their authorship patterns, while age has a more significant effect in the least collaboration-intensive field (Natural Resources)."
2593028313,Measuring scientific impact beyond academia: An assessment of existing impact metrics and proposed improvements,"How does scientific research affect the world around us? Being able to answer this question is of great importance in order to appropriately channel efforts and resources in science. The impact by scientists in academia is currently measured by citation based metrics such as h-index, i-index and citation counts. These academic metrics aim to represent the dissemination of knowledge among scientists rather than the impact of the research on the wider world. In this work we are interested in measuring scientific impact beyond academia, on the economy, society, health and legislation (comprehensive impact). Indeed scientists are asked to demonstrate evidence of such comprehensive impact by authoring case studies in the context of the Research Excellence Framework (REF). We first investigate the extent to which existing citation based metrics can be indicative of comprehensive impact. We have collected all recent REF impact case studies from 2014 and we have linked these to papers in citation networks that we constructed and derived from CiteSeerX, arXiv and PubMed Central using a number of text processing and information retrieval techniques. We have demonstrated that existing citation-based metrics for impact measurement do not correlate well with REF impact results. We also consider metrics of online attention surrounding scientific works, such as those provided by the Altmetric API. We argue that in order to be able to evaluate wider non-academic impact we need to mine information from a much wider set of resources, including social media posts, press releases, news articles and political debates stemming from academic work. We also provide our data as a free and reusable collection for further analysis, including the PubMed citation network and the correspondence between REF case studies, grant applications and the academic literature."
2054096179,Characterizing and Modeling Citation Dynamics,"Citation distributions are crucial for the analysis and modeling of the activity of scientists. We investigated bibliometric data of papers published in journals of the American Physical Society, searching for the type of function which best describes the observed citation distributions. We used the goodness of fit with Kolmogorov-Smirnov statistics for three classes of functions: log-normal, simple power law and shifted power law. The shifted power law turns out to be the most reliable hypothesis for all citation networks we derived, which correspond to different time spans. We find that citation dynamics is characterized by bursts, usually occurring within a few years since publication of a paper, and the burst size spans several orders of magnitude. We also investigated the microscopic mechanisms for the evolution of citation networks, by proposing a linear preferential attachment with time dependent initial attractiveness. The model successfully reproduces the empirical citation distributions and accounts for the presence of citation bursts as well."
2133894160,"Knowledge, networks and nations: global scientific collaboration in the 21st century","Science is a global enterprise. Today there are over 7 million researchers around the world,drawing on a combined international R&D spend of over US$1000 billion(a 45%increase since 2002), and reading and publishing in around 25,000 separate scientific journals per year. These researchers collaborate with each other,motivated by wishing to work with the very best people and facilities in the world, and by curiosity, seeking new knowledge to advance their field or to tackle specific problems. Knowledge, Networks and Nations reviews, based on available data,the changing patterns of science, and scientific collaboration, in order to provide a basis for understanding such ongoing changes. It aims to identify the opportunities and benefits of international collaboration,to consider how they can best be realised, and to initiate a debate on how international scientific collaboration can be harnessed to tackle global problems more effectively. From Singapore to South Africa, new researchers and research communities are reshaping the landscape for science and innovation, so long dominated by the USA, Japan and Europe. This report explores this changing geography of science and innovation."
2164152683,Approaching the “reward triangle”: General analysis of the presence of funding acknowledgments and “peer interactive communication” in scientific publications,"Understanding the role of acknowledgments given by researchers in their publications has been a recurrent challenge in the bibliometric field, but relatively unexplored until now. This study presents a general bibliometric analysis on the new “funding acknowledgment” (FA) information available in the W eb of S cience. All publications covered by the database in 2009 have been analyzed. The presence and length of the FA text, as well as the presence of “peer interactive communication” in the acknowledgments, are related to impact indicators, distribution of papers by fields, countries of the authors, and collaboration level of the papers. It is observed that publications with FAs present a higher impact as compared with publications without them. There are also differences across countries and disciplines in the share of publications with FAs and the acknowledgment of peer interactive communication. C hina is the country with the highest share of publications acknowledging funding, while the presence of FAs in the humanities and social sciences is very low compared to the more basic disciplines. The presence of peer interactive communication in acknowledgments can be linked to countries that have a strong scientific tradition and are incorporated in scientific networks. Peer interactive communication is also common in the fields of humanities and social sciences and can be linked to lower levels of co‐authorship. Observed patterns are explained and topics of future research are proposed."
2230234779,An Analysis of Altmetrics in Emergency Medicine,"Alternative-level metrics (Altmetrics) are a new method to assess the sharing and spread of scientific knowledge. The primary objective of this study was to describe the traditional metrics and Altmetric scores of the 50 most frequently cited articles published in emergency medicine (EM) journals. Since many articles related to EM are published in other journals, the secondary aim of this study was to describe the Altmetric scores of the most frequently cited articles relevant to EM in other biomedical journals.A structured search of the Institute for Scientific Information Web of Science version of the Science Citation Index Expanded was conducted. The 200 most frequently cited articles in the top 10 EM journals (2011 Journal Citation Report) were identified. The 200 most frequently cited articles from the rest of the medical literature, matching a predefined list of keywords relevant to the specialty of EM, were identified. Two authors reviewed the lists of citations for relevance to EM and a consensus approach was used to arrive at the final lists of the top 50 cited articles. The Altmetric scores for the top 50 cited articles in EM and other journals were determined. Descriptive statistics and Spearman correlation were performed.The highest Altmetric score for EM articles was 25.0; the mean (±SD) was 1.9 (±5.0). The EM journal with the highest mean article Altmetric score was Resuscitation. The main clinical areas shared for articles from EM articles were trauma (mean ± SD = 11.0 ± 15.6, median = 11.0) and cardiac arrest (mean ± SD = 2.7 ± 5.8, median = 0). The highest Altmetric score for other journals was 176.0 (mean ± SD = 23.3 ± 40.8). The other journal with the highest mean article Altmetric score was the New England Journal of Medicine. The main clinical areas shared for articles were critical care (mean ± SD score = 36.5 ± 47.4, median = 36.5), sepsis (mean ± SD = 24.6 ± 48.8, median = 12.0), cardiology (mean ± SD = 19.2 ± 35.6, median = 7.0), and infectious diseases (mean ± SD = 17.0 ± 12.7, median = 17.0). Spearman correlation demonstrated weakly positive correlation between citation counts and Altmetric scores for EM articles and other journals.This study is the first analysis of Altmetric scores for the top cited articles in EM. We demonstrated that there is a mild correlation between citation counts and Altmetric scores for the top papers in EM and other biomedical journals. We also demonstrated that there is a gap between the sharing of the top articles in EM journals and those related to EM in other biomedical journals. Future research to explore this relationship and its temporal trends will benefit the understanding of the reach and dissemination of EM research within the scientific community and society in general."
4211168474,Web Indicators for Research Evaluation: A Practical Guide,"In recent years there has been an increasing demand for research evaluation within universities and other research-based organisations. In parallel, there has been an increasing recognition that tradi"
4378220440,Gauging the research performance of BRICS in the domain of Library and Information Science through Performance analysis and Science mapping,"The study gauges the research performance of the BRICS bloc in the field of Library and Information Science (LIS) research using Performance analysis and Science mapping. The Web of Science database is used for the study and articles published between 2013 and 2022 have been selected for analysis. Data analysis and visualisation have been done using the Bibliometrix R package and VOSviewer. The findings reveal an upward trend in publications. Furthermore, China has been the most prolific nation in terms of productivity and impact. Scientometrics is the leading source in terms of publications while the International Journal of Information Management is the most cited source. With regard to author productivity, Zhang Y has the highest number of publications while Lowry PB is the most cited author. Wuhan University is the most productive organization. In terms of collaboration, the USA is the primary partner for the entire BRICS group, particularly China and collaboration among the BRICS isn’t as significant as it is with the non-BRICS countries. This study provides insightful information about recent scientific developments in the field of LIS. Additionally, by using this research as a guide, researchers from different fields will be able to analyse how the body of knowledge on a certain subject has evolved over time. This study also outlines potential research directions in this field of research."
1624797311,Living with the H-Index? Metric Assemblages in the Contemporary Academy,"This paper examines the relationship between metrics, markets and affect in the contemporary UK academy. It argues that the emergence of a particular structure of feeling amongst academics in the last few years has been closely associated with the growth and development of ‘quantified control’. It examines the functioning of a range of metrics: citations; workload models; transparent costing data; research assessments; teaching quality assessments; and commercial university league tables. It argues that these metrics, and others, although still embedded within an audit culture, increasingly function autonomously as a data assemblage able not just to mimic markets but, increasingly, to enact them. It concludes by posing some questions about the possible implications of this for the future of academic practice."
3002741938,Создание научной репутации в цифровой век: опосредованный подход. Обзорная статья,"Представляется исследование, цель которого понять, как сегодня ученые могут, и на самом деле должны подходить к созданию, сохранению и распространению своей профессиональной репутации. Предлагаемый здесь обзор литературы анализирует доступные ученым репутационные возможности, имеющиеся во все больше расширяющейся, основанной на ценностях, цифровой и сетевой среде Science 2.0. Фокусируясь на концептуальном подходе, разработанном при участии Европейской комиссии и непосредственно ориентированном на проведение аналитических обзоров и аудита репутационных ценностей в научной деятельности, данное исследование достаточно подробно рассматривает более 30 практик, включающих современное научное предприятие, с точки зрения завоевания репутации."