This post is coauthored by Tim Vines and Ben Kaube. Tim is the founder and CEO of DataSeer. Ben is a cofounder of Cassyni, a platform for helping publishers and institutions create and engage communities of researchers using seminars. It originally appeared on The Scholarly Kitchen blog April 16, 2024.
Assessing research integrity issues for articles submitted to an academic journal is a daunting prospect: the journal must somehow use the submitted text, figures, and (if available) accompanying data to uncover unethical behavior by the authors. This behavior (such as fabricating or altering data, etc.) may have taken place long before the article itself was written.
However, these methods primarily focus on detecting bad behavior. To complement this approach, there’s a compelling need for indicators of “honest signaling” in research — a concept that, instead of aiming to catch bad behavior, captures commitment to ethical research practices.
It is instructive to draw analogy to the evolutionary biology concept known as “honest signaling.” This principle illustrates how certain traits in the animal kingdom, such as the peacock’s extravagant tail or the bright colors of some jumping spiders, serve as ‘honest’ markers of the individual’s fitness. These traits are energetically costly to produce and maintain, signifying that only individuals in prime condition can afford these displays. Thus, these signals are a reliable indicator of an individual’s fitness and hence their quality as a mate.
Translating this concept to the realm of research integrity, we seek analogous signals within researcher behavior — practices that are sufficiently demanding of time, effort, or resources, such that they are unlikely to be undertaken by those not genuinely committed to ethical research. Honest signaling in this context would encompass activities that are both visible and verifiable, shedding light on the researcher’s dedication to transparency and integrity:
- Open Science Practices: This includes the sharing of raw data, pre-registration of studies, publishing of preprints, and sharing of the code and scripts to operate command line software. These practices not only require a significant investment of time and resources but also open the researcher’s work to scrutiny and verification by the wider community.
- Community Engagement: Active participation in the scientific community through presentations at conferences, seminars, and poster sessions acts as a form of honest signaling. Such engagements are not only resource-intensive, but also expose one’s research to critique and validation by peers. Moreover, maintaining a visible online presence (e.g., personal webpages and ORCID profiles) and contributions to journal publishing outside of authorship roles (e.g., writing peer reviews and membership on editorial boards) further exemplifies a researcher’s commitment to open discourse and collaboration.
These signals, akin to the peacock’s tail, are not just markers of the quality and credibility of the individual’s work but also serve to signal their commitment to the principles of research integrity.
Any indicators of honest signaling should be considered contextually, acknowledging that open research practices differ between fields and engagement with scholarly communities varies regionally. This is particularly true when considering researchers facing systemic barriers such as those in the Global South. This suggests that simple scores of honest signaling will be overly reductive and that honest signaling indicators should be evaluated qualitatively as part of the editorial process.
The continued growth of digital platforms and the increase in online data post-COVID offer new opportunities to operationalize honest signaling indicators. By aggregating data from society meetings, departmental web pages, and conference programs, it becomes possible to capture engagement of researchers within their respective fields at scale and produce a suite of indicators for research integrity purposes (e.g., what is the publication record of author X? have they presented similar work at relevant conferences or seminar series? Does the manuscript have accompanying data or code?).
One important consideration when aggregating data from many venues is the difficulty of author disambiguation. ORCID has an important role to play here, but the usual challenge of preventing false positives is compounded by the fact that many research outputs associated with honest signaling have traditionally lacked persistent identifiers and high quality metadata. For example posters and seminar recordings have only recently started receiving persistent identifiers.
In biology, honest signals are an indicator of an individual’s fitness, and while some of those signals – like the peacock’s tail – might be eye-catching to us, they are fine tuned to be evaluated by individuals in the same species. Similarly, in the world of research, it will always be the experts in one’s own field that will be best positioned to pick up on and assess honest signaling.
The concept of honest signaling offers a promising framework for verifying research integrity in a manner that complements existing tools aimed at detecting misconduct. By focusing on behaviors that are costly in terms of time and effort and indicate a researcher’s commitment to ethical practices, this approach not only helps identify instances of integrity but also encourages a culture of openness and engagement within the scientific community. Operationalizing indicators of “honest signaling”, while challenging, presents an opportunity to foster transparency and accountability in academic research, ensuring that it remains a pursuit marked by ethical conduct and genuine discovery.
