Search

We examine the hypothesis that research collaboration has enabled a global research network to evolve, with self-organizing properties transcending national research policy. We examine research output, bilateral and multilateral collaboration, subject diversity, and citation impact over 40 years, in detail for the G7 and BRICK groups of countries and in summary for 26 other nations. We find that the rise in national output was strongly associated with bilateral collaboration until the 2000s but after that by multilateral partnerships, with the shift happening at much the same time across countries. There was a general increase in research subject diversity, with evenness across subjects converging on a similar index value for many countries. Similar diversity is not the same as actual similarity but, in fact, the G7 countries became increasingly similar. National average citation impact (CNCI) rose and groups converged on similar impact values. The impact of the largest economies is above world average, which is a phenomenon we discuss separately. The similarities in patterns and timing occur across countries despite variance in their research policies, such as research assessment. We suggest that the key agent facilitating global network self-organization is a shared concept of best practice in research.

In December 2020, the second edition of the Handbook for Scientometrics: Science and Technology Development Indicators will be published. Over the six years since the release of the first edition, interest in scientometrics and its practical application has noticeably grown in Russia, and Russian centers for scientometrics have appeared. The introduction for the new edition is of interest itself. The Russian version is published with the kind permission of the copyright holder, Clarivate Analytics.

National research diversity is explored through the balance of global and national papers in journal categories in the Web of Science (WoS) and Essential Science Indicators (ESI) and we examine the consequences of “normalizing” national publication counts against global baselines. Global balance across subject categories became more even as annual WoS indexing grew fourfold between 1981 and 2018, with a relative shift from biomedicine towards environment and technology. Change at the country level may have tracked this or been influenced by local policy and funding. We discuss choice of methods and indices for analysis: WoS categories provide better granularity than ESI; Lorenz curves are explored but found limiting; the Pratt index, Gini coefficient, and Shannon diversity are compared. At the national level, balance generally increases and is greatest in non-Anglophone countries, perhaps due to shifts in language and journal use. Two aspects of national change are revealed: the balance of actual WoS paper counts and the balance of counts normalized against world baseline. The broad patterns for these analyses are similar, but normalized data indicate relatively greater evenness. National patterns link to research capacity and regional networking opportunities, while international collaboration may blend national differences. A data set is provided for analytical use.

While bibliometric analysis is normally able to rely on complete publication sets this is not universally the case. For example, Australia (in ERA) and the UK (in the RAE/REF) use institutional research assessment that may rely on small or fractional parts of researcher output. Using the Category Normalised Citation Impact (CNCI) for the publications of ten universities with similar output (21,000–28,000 articles and reviews) indexed in the Web of Science for 2014–2018, we explore the extent to which a ‘sample’ of institutional data can accurately represent the averages and/or the correct relative status of the population CNCIs. Starting with full institutional data, we find a high variance in average CNCI across 10,000 institutional samples of fewer than 200 papers, which we suggest may be an analytical minimum although smaller samples may be acceptable for qualitative review. When considering the ‘top’ CNCI paper in researcher sets represented by DAIS-ID clusters, we find that samples of 1000 papers provide a good guide to relative (but not absolute) institutional citation performance, which is driven by the abundance of high performing individuals. However, such samples may be perturbed by scarce ‘highly cited’ papers in smaller or less research-intensive units. We draw attention to the significance of this for assessment processes and the further evidence that university rankings are innately unstable and generally unreliable.

Citations can be an indicator of publication significance, utility, attention, visibility or short-term impact but analysts need to confirm whether a high citation count for an individual is a genuine reflection of influence or a consequence of extraordinary, even excessive, self-citation. It has recently been suggested there may be increasing misrepresentation of research performance by individuals who self-cite inordinately to achieve scores and win rewards. In this paper we consider self-referencing and self-citing, describe the typical shape of self-citation patterns for carefully curated publication sets authored by 3517 Highly Cited Researchers and quantify the variance in the distribution of self-citation rates within and between all 21 Essential Science Indicators’ fields. We describe both a generic level of median self-referencing rates, common to most fields, and a graphical, distribution-driven assessment of excessive self-citation that demarcates a threshold not dependent on statistical tests or percentiles (since for some fields all values are within a central ‘normal’ range). We describe this graphical procedure for identifying exceptional self-citation rates but emphasize the necessity for expert interpretation of the citation profiles of specific individuals, particularly in fields with atypical self-citation patterns.

This paper presents an analysis of the Overton policy document database, describing the makeup of materials indexed and the nature in which they cite academic literature. We report on various aspects of the data, including growth, geographic spread, language representation, the range of policy source types included, and the availability of citation links in documents. Longitudinal analysis over established journal category schemes is used to reveal the scale and disciplinary focus of citations and determine the feasibility of developing field-normalized citation indicators. To corroborate the data indexed, we also examine how well self-reported funding outcomes collected by UK funders correspond to data indexed in the Overton database. Finally, to test the data in an experimental setting, we assess whether peer-review assessment of impact as measured by the UK Research Excellence Framework (REF) 2014 correlates with derived policy citation metrics. Our findings show that for some research topics, such as health, economics, social care, and the environment, Overton contains a core set of policy documents with sufficient citation linkage to academic literature to support various citation analyses that may be informative in research evaluation, impact assessment, and policy review.

Many academic analyses of good practice in the use of bibliometric data address only technical aspects and fail to account for and appreciate user requirements, expectations, and actual practice. Bibliometric indicators are rarely the only evidence put before any user group. In the present state of knowledge, it is more important to consider how quantitative evaluation can be made simple, transparent, and readily understood than it is to focus unduly on precision, accuracy, or scholarly notions of purity. We discuss how the interpretation of ‘performance’ from a presentation using accurate but summary bibliometrics can change when iterative deconstruction and visualization of the same dataset is applied. From the perspective of a research manager with limited resources, investment decisions can easily go awry at governmental, funding program, and institutional levels. By exploring select real-life data samples we also show how the specific composition of each dataset can influence interpretive outcomes.

Researcher behaviour is shown to change under assessment. An unexpected time-skew towards most recent papers in each census period was found among the outputs selected by UK academics for the research assessment cycles of the 1990s. This skew changed to a more even time-based distribution for scientists and engineers in later cycles. At the same time engineers switched their preferred output type for submission, from conference proceedings to journal articles. Social scientists also switched, from monographs to journal articles. There was no discussion of these output patterns at the time, or later, but the patterns and their evolution had marked consistency across subjects and institutions. These changes are discussed in terms of consensus and influences on researcher concepts of the evidence of excellence. The increasing availability of citation data in the 1990s and the likely role of citation analysis as a steering factor is noted.

Greater collaboration generally produces higher category normalised citation impact (CNCI) and more influential science. Citation differences between domestic and international collaborative articles are known, but obscured in analyses of countries’ CNCIs, compromising evaluation insights. Here, we address this problem by deconstructing and distinguishing domestic and international collaboration types to explore differences in article citation rates between collaboration type and countries. Using Web of Science article data covering 2009–2018, we find that individual country citation and CNCI profiles vary significantly between collaboration types (e.g., domestic single institution and international bilateral) and credit counting methods (full and fractional). The ‘boosting’ effect of international collaboration is greatest where total research capacity is smallest, which could mislead interpretation of performance for policy and management purposes. By incorporating collaboration type into the CNCI calculation, we define a new metric labelled Collab-CNCI. This can account for collaboration effects without presuming credit (as fractional counting does). We recommend that analysts should: (1) partition all article datasets so that citation counts can be normalised by collaboration type (Collab-CNCI) to enable improved interpretation for research policy and management; and (2) consider filtering out smaller entities from multinational and multi-institutional analyses where their inclusion is likely to obscure interpretation.

OpenVIVO is a free and open-hosted semantic web platform that anyone can join and that gathers and shares open data about scholarship in the world. OpenVIVO, based on the VIVO open-source platform, provides transparent access to data about the scholarly work of its participants. OpenVIVO demonstrates the use of persistent identifiers, the automatic real-time ingest of scholarly ecosystem metadata, the use of VIVO-ISF and related ontologies, the attribution of work, and the publication and reuse of data—all critical components of presenting, preserving, and tracking scholarship. The system was created by a cross-institutional team over the course of 3 months. The team created and used RDF models for research organizations in the world based on Digital Science GRID data, for academic journals based on data from CrossRef and the US National Library of Medicine, and created a new model for attribution of scholarly work. All models, data, and software are available in open repositories.

The need to improve population monitoring and enhance surveillance of infectious diseases has never been more pressing. Factors such as air travel act as a catalyst in the spread of new and existing viruses. The unprecedented user-generated activity on social networks over the last few years has created real-time streams of personal data that provide an invaluable tool for monitoring and sampling large populations. Epidemic intelligence relies on constant monitoring of online media sources for early warning, detection, and rapid response; however, the real-time information available in social networks provides a new paradigm for the early warning function. The communication of risk in any public health emergency is a complex task for governments and healthcare agencies. This task is made more challenging in the current situation when the public has access to a wide range of online resources, ranging from traditional news channels to information posted on blogs and social networks. Twitter’s strength is its two-way communication nature --- both as an information source but also as a central hub for publishing, disseminating and discovering online media. This study addresses these two challenges by investigating the role of Twitter during the 2009 swine flu pandemic by analysing data collected from the SN, and by Twitter using the opposite way for dissemination information through the network. First, we demonstrate the role of the social network for early warning by detecting an upcoming spike in an epidemic before the official surveillance systems by up to two weeks in the U.K. and up to two to three weeks in the U.S. Second, we illustrate how online resources are propagated through Twitter at the time of the WHO’s declaration of the swine flu “pandemic”. Our findings indicate that Twitter does favour reputable t bogus information can still leak into the network.

Background Social media promotion is increasingly adopted by organizers of industry and academic events; however, the success of social media strategies is rarely questioned or the real impact scientifically analyzed. Objective We propose a framework that defines and analyses the impact, outreach, and effectiveness of social media for event promotion and research dissemination to participants of a scientific event as well as to the virtual audience through the Web. Methods Online communication channels Twitter, Facebook, Flickr, and a Liveblog were trialed and their impact measured on outreach during five phases of an eHealth conference: the setup, active and last-minute promotion phases before the conference, the actual event, and after the conference. Results Planned outreach through online channels and social media before and during the event reached an audience several magnitudes larger in size than would have been possible using traditional means. In the particular case of eHealth 2011, the outreach using traditional means would have been 74 attendees plus 23 extra as sold proceedings and the number of downloaded articles from the online proceedings (4107 until October 2013). The audience for the conference reached via online channels and social media was estimated at more than 5300 in total during the event. The role of Twitter for promotion before the event was complemented by an increased usage of the website and Facebook during the event followed by a sharp increase of views of posters on Flickr after the event. Conclusions Although our case study is focused on a particular audience around eHealth 2011, our framework provides a template for redefining “audience” and outreach of events, merging traditional physical and virtual communities and providing an outline on how these could be successfully reached in clearly defined event phases.

We report on preliminary analysis on user engagement in two online medical communities of practice. Despite the communities being independent of each other, and dealing with different domains (field epidemiology and therapeutic prescribing for optometrists), there are some clear similarities in the networks of users, and in patterns of replies to user postings. We also draw some initial conclusions to help maintain user engagement in these and similar sites, and we suggest some future lines of research.

Much attention has been focused on Twitter because it serves as a central hub for the publishing, dissemination, and discovery of online media. This is true for both traditional news outlets and user generated content, both of which can vary widely in their journalistic and scientific quality. The recent Swine Flu pandemic of 2009 highlighted this aspect perfectly, global events that created a large online buzz, with some dubious medical facts leaking into public opinion. This paper presents an investigation into how online resources relating to Swine Flu were discussed on Twitter, with a focus on identifying and analyzing the popularity of trusted information sources (e.g. from quality news outlets and official health agencies). Our findings indicate that reputable sources are more popular than untrusted ones, but that information with poor scientific merit can still leak into to the network and potentially cause harm.

While an ever increasing popularity of online wiki platforms, user-tagging tools, blogs, and forums is the core characteristic of the Web 2.0 era, converting an existing high-quality training module into a collaborative online space for an active community of practice (CoP) while preserving its quality approval processes is a challenging task. This is the aim of the ECDC-funded Field Epidemiology Manual (FEM) wiki project, based on training resources organized in 17 chapters developed for the European EPIET epidemiology training programme. This paper describes the challenges, solutions, and development processes behind the FEM wiki portal – an online collaborative Web 2.0 platform taking advantage of the user-generated input while preserving the structure, editorial processes and style of the existing FEM manual. We describe the need for ECDC-recognised content and discuss the editorial roles developed in this European project but applicable to any other training resource converted into an online wiki platform.

Early warning systems for the identification and tracking of infections disease outbreaks have become an important tool in the field of epidemiology. While government lead initiatives to increase the sharing of surveillance data have improved early detection and control, along with advanced web monitoring and analytics services, the recent swine flu outbreak of 2009 demonstrated the important role social media has and the wealth of data it exposes. In this paper, we present an investigation into Twitter, using around 3 Million tweets gathered between May and December 2009, as a possible source of surveillance data and its feasibility to serve as an early warning system. By performing simple filtering and normalization, we demonstrate that Twitter can serve as a self-reporting tool, and hence, provide indications of increased infection spreading. Our initial findings indicate that Twitter can detect such events up to one week before conventional GP reported surveillance data.

International audience; Popularity and spread of online social networking in recent years has given a great momentum to the study of dynamics and patterns of social interactions. However, these studies have often been confined to the online world, neglecting its interdependencies with the offline world. This is mainly due to the lack of real data that spans across this divide. The Live Social Semantics application is a novel platform that dissolves this divide, by collecting and integrating data about people from (a) their online social networks and tagging activities from popular social networking sites, (b) their publications and co-authorship networks from semantic repositories, and (c) their real-world face-to-face contacts with other attendees collected via a network of wearable active sensors. This paper investigates the data collected by this application during its deployment at three major conferences, where it was used by more than 400 people. Our analyses show the robustness of the patterns of contacts at various conferences, and the influence of various personal properties (e.g. seniority, conference attendance) on social networking patterns.

Explicit and implicit feedback exhibits different characteristics of users' preferences with both pros and cons. However, a combination of these two types of feedback provides another paradigm for recommender systems (RS). Their combination in a user preference model presents a number of challenges but can also overcome the problems associated with each other. In order to build an effective RS on combination of both types of feedback, we need to have comparative data allowing an understanding of the computation of user preferences. In this paper, we provide an overview of the differentiating characteristics of explicit and implicit feedback using datasets mined from Last.fm, an online music station and recommender service. The datasets consisted of explicit positive feedback (by loving tracks) and implicit feedback which is inherently positive (the number of times a track is played). Rather than relying on just one type of feedback, we present techniques for extracting user preferences from both. In order to compare and contrast the performances of these techniques, we carried out experiments using the Taste recommender system engine and the Last.fm datasets. Our results show that implicit and explicit positive feedback complements each other, with similar performances despite their different characteristics.

In this paper, we present our study and characterisation of explicit and implicit feedback on Last.fm, an online music station and recommender service. The dataset consisted of explicit positive feedback (through loved tracks) and implicit positive feedback (the number of times a track is played). As one would expect, our analysis shows that explicit feedback is very scarce. However, we also found that the rate at which a user provides explicit feedback decreases with time, and that overall leaving explicit feedback has a negative effect on the user's behaviour.

This paper describes an Open Linked Data back- linking service, a generic architecture component to support the discovery of useful links between items across highly connected data sets. Using Public Sector Information (PSI) currently available as Linked Data, we demonstrate that contemporary publishing practices do not adequately support the ability to navigate or automatically traverse between resources published by different vendors, or the capacity to discover information relevant to a particular URI. Although some useful services in this area have been developed, such as large triple indexes of published data, and the collection of sameAs relationships between individuals, we believe that an important component is missing: a mechanism to discover the backlinks to relevant resources that cannot be found by direct URI resolution. We present the implementation of such a component, integrating data from various PSI sources.

International audience; The Live Social Semantics is an innovative application that encourages and guides social networking between researchers at conferences and similar events. The application integrates data from the Semantic Web, online social networks, and a face-to-face contact sensing platform. It helps researchers to find like-minded and influential researchers, to identify and meet people in their community of practice, and to capture and later retrace their real-world networking activities. The application was successfully deployed at two international conferences, attracting more than 300 users in total. This paper describes the Live Social Semantics application, with a focus on how data from Web 2.0 sources can be used to automatically generate Profiles of Interest. We evaluate and discuss the results of its two deployments, assessing the accuracy of profiles generated, the willingness to link to external social networking sites, and the feedback given through user questionnaires.

International audience; We describe a novel application that integrates real-world data on the face-to-face proximity of individuals with their identities and contacts in on-line social networks. This application was successfully deployed at two conference gatherings, ESWC09 and HT2009, and actively used by hundreds of people. Personal profiles of the participants were automatically generated using several Web 2.0 systems and semantic data sources, and integrated in real-time with face-toface proximity relations detected using RFID-enabled badges. The integration of these heterogeneous data sources enables various services that enhance the experience of conference attendees, allowing them to explore their social neighborhood and to connect with other participants. This paper describes the architecture of the application, the services we provided, and the results we achieved in these deployments.

A single datum or a set of a categorical data has little value on its own. Combinations of disparate sets of data increase the value of those data sets and helps to discover interesting patterns or relationships, facilitating the construction of new applications and services. In this paper, we describe an implementation of using open geographical data as a core set of “join point”(s) to mesh different public datasets. We describe the challenges faced during the implementation, which include, sourcing the datasets, publishing them as linked data, and normalising these linked data in terms of finding the appropriate “join points” from the individual datasets, as well as developing the client application used for data consumption. We describe the design decisions and our solutions to these challenges. We conclude by drawing some general principles from this work.

International audience; Social interactions are one of the key factors to the success of con- ferences and similar community gatherings. This paper describes a novel appli- cation that integrates data from the semantic web, online social networks, and a real-world contact sensing platform. This application was successfully deployed at ESWC09, and actively used by 139 people. Personal profiles of the partici- pants were automatically generated using several Web 2.0 systems and semantic academic data sources, and integrated in real-time with face-to-face contact net- works derived from wearable sensors. Integration of all these heterogeneous data layers made it possible to offer various services to conference attendees to en- hance their social experience such as visualisation of contact data, and a site to explore and connect with other participants. This paper describes the architecture of the application, the services we provided, and the results we achieved in this deployment.

This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in HT '08 Proceedings of the nineteenth ACM conference on Hypertext and hypermedia, http://dx.doi.org/10.1145/1379092.1379103., As the popularity of the web increases, particularly the use of social networking sites and style sharing platforms, users are becoming increasingly connected, sharing more and more information, resources, and opinions. This vast array of information presents unique opportunities to harvest knowledge about user activities and interests through the exploitation of large-scale, complex systems. Communal tagging sites, and their respective folksonomies, are one example of such a complex system, providing huge amounts of information about users, spanning multiple domains of interest. However, the current Web infrastructure provides no mechanism for users to consolidate and exploit this information since it is spread over many desperate and unconnected resources. In this paper we compare user tag-clouds from multiple folksonomies to: (a) show how they tend to overlap, regardless of the focus of the folksonomy (b) demonstrate how this comparison helps finding and aligning the user's separate identities, and (c) show that cross-linking distributed user tag-clouds enriches users profiles. During this process, we find that significant user interests are often reflected in multiple Web2.0 profiles, even though they may operate over different domains. However, due to the free-form nature of tagging, some correlations are lost, a problem we address through the implementation and evaluation of a user tag filtering architecture., This research has been supported by the TAGora project, funded by the Future and Emerging Technologies program (IST-FET) of the European Commission under the contract IST-34721, and the Spanish Ministry of Science and Education (TIN2005–6885). The information provided is the sole responsibility of the authors and does not reflect the Commission’s opinion. The Commission is not responsible for any use that may be made of data appearing in this publication.

Many real-world tasks require the acquisition and integration of information from a distributed set of heterogeneous sources. Hence, there's no shortage of opportunities for applications using Semantic Web (SW) technologies. The power of publishing and linking data in a way that machines can automatically interpret through ontologies is beginning to materialize. However, market penetration level is relatively low, and it's still no routine matter for an enterprise, organization, governmental agency, or business with large distributed databases to add them to the Web of linked and semantically enriched data. In part, they may suspect that they're expected to pioneer an approach in which quick wins are few. Moreover, cost and privacy issues arise when ever-increasing amounts of information are linked into the Web. A practical approach to adopting semantic Web technologies enables large organizations to share data while achieving clear private as well as public reuse benefits.

The final publication is available at Springer via http://dx.doi.org/10.1007/978-3-540-88564-1_40, Proceedings of 7th International Semantic Web Conference, ISWC 2008, Karlsruhe, Germany, October 26-30, 2008., The continued increase in Web usage, in particular participation in folksonomies, reveals a trend towards a more dynamic and interactive Web where individuals can organise and share resources. Tagging has emerged as the de-facto standard for the organisation of such resources, providing a versatile and reactive knowledge management mechanism that users find easy to use and understand. It is common nowadays for users to have multiple profiles in various folksonomies, thus distributing their tagging activities. In this paper, we present a method for the automatic consolidation of user profiles across two popular social networking sites, and subsequent semantic modelling of their interests utilising Wikipedia as a multi-domain model. We evaluate how much can be learned from such sites, and in which domains the knowledge acquired is focussed. Results show that far richer interest profiles can be generated for users when multiple tag-clouds are combined., This research has been supported by the TAGora project funded by the Future and Emerging Technologies program (IST-FET) of the European Commission under the contract IST-34721. The information provided is the sole responsibility of the authors and does not reflect the Commission’s opinion. The Commission is not responsible for any use that may be made of data appearing in this publication

As the Web Services and Grid community adopt Semantic Web technology, we observe a shift towards higherlevel workflow composition and service discovery practices. While this provides excellent functionality to non-expert users, more sophisticated middleware is required to hide the details of service invocation and service integration. An investigation of a common Bioinformatics use case reveals that the execution of high-level workflow designs requires additional processing to harmonise syntactically incompatible service interfaces. In this paper, we present an architecture to support the automatic reconciliation of data formats in such Web Service worklflows. The mediation of data is driven by ontologies that encapsulate the information contained in heterogeneous data structures supplying a common, conceptual data representation. Data conversion is carried out by a Configurable Mediator component, consuming mappings between XML schemas and OWL ontologies. We describe our system and give examples of our mapping language against the background of a Bioinformatics use case.

While the Grid and Web Services have helped us support heterogeneous resource access through the use of service oriented architectures, they have not addressed the issue of heterogeneous data representation. Since service providers often describe their service interfaces using different data models than those assumed by the client, it is common for additional processing to be required to compensate for the mismatch in data formats. By utilising technology from the Semantic Web, we are able to augment existing Web Service systems with middleware to automatically perform data harmonisation when a syntactic mismatch occurs. To achieve this, we have developed a mapping language which can be used to annotate XML data structures with OWL concepts and properties, a Mapping Language Engine to implement this language, and a Dynamic Web Service Invocation component to execute Web Services.

Large-scale, dynamic and open environments such as the Grid and Web Services build upon existing computing infrastructures to supply dependable and consistent large-scale computational systems. This kind of architecture has been adopted by the business and scientific communities allowing them to exploit extensive and diverse computing resources to perform complex data processing tasks. In such systems, results are often derived by composing multiple, geographically distributed, heterogeneous services as specified by intricate workflow management. This leads to the undesirable situation where the results are known, but the means by which they were achieved is not. With both scientific experiments and business transactions, the notion of lineage and dataset derivation is of paramount importance since without it, information is potentially worthless. We address the issue of {\em data provenance\/}, the description of the origin of a piece of data, in these environments showing the requirements, uses and implementation difficulties. We propose an infrastructure level support for a provenance recording capability for service-oriented architectures such as the Grid and Web Services. We also offer services to view and retrieve provenance and we provide a mechanism by which provenance is used to determine whether previous computed results are still up to date.