Max Leckert

Recent years have witnessed a revival of the old question what research quality is and how best to assess it. These debates have been propelled by the intermingling of new metrics and novel value concepts. We first relate researchers' quality conceptions to their appraisal of metric indicators (25 semi-structured interviews). Second, we contrast these views with 14 expert interviews about Altmetrics' perils and potentials. Valuing indicators that reflect scientific chains of production, researchers judge few Altmetrics as useful. Conversely, experts value Altmetrics' potentials to explore new impacts and align evaluation accordingly. Altmetrics exemplify a reterritorialization of research (evaluation) along the lines of platform capitalism.

Expertise in research (e-)valuation is a currently debated matter (CoARA, 2022; DORA, 2012; Wilsdon et al., 2015). The Altmetrics- (Priem et al., 2010) and the Leiden Manifesto (2015) exemplified diametrical visions of how expertise in assessing research should be distributed and practiced (Leckert, 2021). As inherently ambiguous measures, Altmetrics offer insights into how new forms of expertise arise, namely platformized information brokerage. We analyze how multi-dimensional ambivalences around Altmetrics are silenced by motifs of discursive convergence in order to show a) how research assessment is induced with new objectives, and b) how new forms of expertise re-distribute competences and responsibilities in research (e-) valuation.

Bibliometric measures are increasingly used in academic funding and hiring situations. Although rather vaguely described as “impact” or “performance” indicators, they often appear to be equaled with research quality. Altmetrics challenge established bibliometric evaluation concepts by promoting alternative notions of impact: Attention, visibility or societal relevance are some of the concepts that may be used to charge metrics with value and thus add to the many dimensions of quality. Research quality, however, is a hard to define concept. Researchers are likely to express different understandings of quality, influenced by their fields’ shared epistemic and cultural beliefs.

The advance of Open Science by Digital science communication (DSC) has been a major argument in promoting Altmetrics. Such metrifications of DSC have turned out to be a profitable business: Collected from commercial online platforms, aggregated by for-profit companies, showcased by most large publishers. While the question "what do they mean" remains, Altmetrics are reflected back on scientists, scoring their work alongside citation measures. By analyzing interviews with both researchers active in DSC and experts from the Altmetrics field, this contribution contrasts scientists' situated online behavior with its re(e-)valuation in aggregated form. This allows to study the production of reactive potentials in science (e-) valuation.

The presented article comparatively analyzes two manifestos in the field of quantitative science evaluation, the Altmetrics Manifesto (AM) and the Leiden Manifesto (LM). It employs perspectives from the Sociology of (E-) Valuation to make sense of highly visible critiques that organize the current discourse. Four motifs can be reconstructed from the manifestos’ valuation strategies. The AM criticizes the confinedness of established evaluation practices and pledges for an expansion of quantitative research evaluation. The LM denounces the proliferation of ill-applied research metrics and calls for an enclosure of metric research assessment. It can be shown that these motifs are organized diametrically.