Dr. Stephan Stahlschmidt

In this study, we investigated whether citation context information is able to increase the validity of citation impact analyses to measure research quality compared to simple citation counts. We analyzed the statistical relationships of information extracted from structured citation context data in the Web of Science (Clarivate) such as the placement of citations within specific sections of an article with post-publication peer review quality ratings from Faculty Opinions (H1 connect), used as an external validity criterion for research quality. The study is based on publications in medicine and life sciences. [...]

The presence or absence of publishers in large bibliographic databases highlights tangible and symbolic differences between small and large publishers, underlining their diverging incentive structures. To assess these differences, this contribution explores reference data from small (open access) publishers and their journals in the Web of Science and Scopus. The analysis focuses on references that are not indexed in these databases and examines them from a text-as-data approach. The study frames two dimensions of relevance - visibility and impact - as proxies for assessing the scholarly bibliodiversity represented by small (open access) publishers.

This work proposes using the feature of large language models (LLMs) to learn about the "data bias" injected by diverse SDG classifications into bibliometric data by exploring five SDGs. We build a LLM that is fine-tuned in parallel by the diverse SDG classifications inscribed into the databases' SDG classifications. Our results show high sensitivity in model architecture, classified publications, fine-tuning process and natural language generation. The wide arbitrariness at different levels raises concerns about using LLM in research practice.