I write this column after my return from Brussels, where I was once again involved in the evaluation of research projects conducted by the ERC (European Research Council), the scientific agency of the European Union.
Each year, the ERC opens calls in four ways. One, called “Synergy,” funds collaboration between up to four researchers, one of whom may be on another continent. The other three – “Beginning”, “Embedded” and “Advanced” – support individual projects carried out in Europe by researchers at different stages of their careers. The “grant” (funding) can reach 2.5 million euros (15 million Brazilian reals) per researcher for a period of five years.
I started participating in the “advanced” experiment in 2019, and since 2023 I have been working as a coordinator for the Mathematics Committee. I accepted the invitation out of professional curiosity: having been responsible for scientific evaluation at CNPq, Capes and Faperj, I wanted to know how the main global agencies work (I have also collaborated with the US National Science Foundation), to understand what we can learn from them. After six years, it was worth it.
A major tension in this area concerns the use of “scientific metrics”: to what extent can the evaluation of research rely on quantitative measures such as the number of citations to scientific articles or the impact factor of the journals in which they are published?
The European agency’s position is crystal clear: like the National Science Foundation, the European Research Agency has formally adhered to DORA (San Francisco Declaration on Research Evaluation), which prohibits the use of such metrics, stating that “the scientific content of the article is much more important than the publication criteria or the name of the scientific journal in question.” Therefore, their judgments are based exclusively on qualitative (subjective!) Evaluations of the content of each project carried out by famous experts. This is how the European Research Council has built a highly respected standard of excellence, one of the rare consensuses in the complex European scenario.
More than 3,500 scientific institutions around the world have already joined the course, and it is time for Brazilian scientific bodies to follow suit (the Serrabelheira Institute is already one of the signatories). Why does it matter? Judging scientific content requires more work than using numerical scales. But it is necessary to ensure high-level scientific evaluation, which is an indispensable condition for the progress of science.
Are we moving in the right direction? Years ago, when I was coordinating mathematics at CNPq, the chair, Professor Ernie Camargo, praised the fact that we were in the most qualitative field in our assessments, while ironically some fields in the humanities used purely quantitative criteria. “Mathematicians know very well what numbers cannot do for us,” I explained to him.
But since then, all fields of knowledge have been urged to give priority to so-called “objective” criteria, even though they have nothing to do with the significance of the proposed research. From the desire for supposed “transparency” to the specter of legalization of trials, there are many pressures to evaluate the feasibility of research through numerical rules, which are linked much more to the quantity of production than to the quality of the scientific project.
There are ongoing efforts to try to mitigate this harmful trend for Brazilian science. But the road is long and involves broad commitment to the Dora principles.
Current link: Did you like this text? Subscribers can access seven free accesses from any link per day. Just click on the blue letter F below.