Are You in Australia's Top 250 Researchers? Who Cares!

Australian science had a Very Bad, Terrible week.

On Tuesday CSIRO, Australia’s national science agency, announced it was looking to cut up to 350 jobs. Certain research areas are being de-prioritized. Agricultural and food science, health and biosecurity and the environment unit taking huge hits, according to The Guardian.

The announcement came on the same day the Australian newspaper dropped its annual “Research Magazine”, which, according to the paper, “offers a deep and detailed look at how our nation’s valuable reservoir of knowledge is created”.

Here’s the webpage headline:

The 2026 Research magazine showcases the benefits of research

I’m really glad that we’re covering science! A whole magazine for it! One that’s been running eight years! But beyond this headline, which feels incredible uninspiring, ranking researchers and universities is just an outdated mode of assessing impacts and it’s mostly based on easily manipulated metrics.

For instance, lets take CSIRO. The organisation, or a researcher from the organisation features in just 11 of the 500 results in the magazine (the top 250 researchers + top 250 research institutions by field if the math ain’t mathin’ for you). Birds, forestry and insects are all covered best by CSIRO, according to this ranking, as is physics, zoology and oceanography. But CSIRO is a tiny portion of the pie…

I’m not drawing a line between the cuts at CSIRO and how little of its research made the list — indeed, if the message you take from this newsletter is that one, then I’ve done a Terrible Job.1 What’s more important is trying to understand why we would rank researchers like this at all, what the benefits are and how much it matters.

So first, here’s the methodology as stated on the website to find the top 250 researchers in Australia:

We give every Australian researcher and every Australian university or research organisation an impact score in each of 250 fields of research. This score is equal to the number of citations for papers published (by the individual or the institution) in the top 20 journals of each particular field in the past five years. The researcher and the institution with the top score is the leader of the field. This approach ensures that only quality work, and recent work, is considered. We use publicly available information on Google Scholar to identify researchers, to obtain data on their citations and to link their work with universities and research institutions.

I am not sure how the 250 fields are picked, nor what constitutes the top 20 journals by field.2 And I am a little skeptical that Google Scholar provides the best citation information.3

The magazine itself (here) provides a little more context, suggesting the score is designed to reward research quality, impact and volume.

“Our purpose is to shine a light on Australia’s best research and we delve down into the nitty gritty, highlighting praiseworthy pockets of excellence which might otherwise go unnoticed by the wider community.”

This notion of best is obviously a tricky thing to capture and perhaps we should celebrate any attempt that spotlights Australian research. In this, I would not fault the Australian. But the idea behind this research is not something that fosters scientific excellence. Rather, it creates a research environment that is defined by publication metrics, citation counts and journal quality, while refusing to acknowledge the pressure that these metrics put researchers under.

To highlight this point, Education Minister Jason Clare opens the magazine with this proclamation about Australia’s most recent Nobel Prize winner, Richard Robson:

What drove Professor Robson wasn’t accolades or outcomes – it was curiosity, and that curiosity has led to him receiving one of science’s highest honours. This was homegrown research, undertaken at the University of Melbourne and supported by the Australian Research Council (ARC).

But this notion is at odds with the entire idea of the magazine. By ranking our researchers based on their publications — mostly around the quantity and citation metrics — we are not rewarding researcher’s curiosity. We’re not following in the footsteps of what drove Professor Robson to win a Nobel Prize here.

And Professor Robson, no longer actively researching, is trotted out in this magazine in Clare’s opening story and later in a (very well written and insightful) profile. But Robson, of course, does not make the Top 250 researchers list this year (the methodology doesn’t allow that) and one might assume he would not have been profiled at all if he had not won the Nobel.

What means more? Metrics or Nobels?

Citation Nation

I am more interested in where the list breaks down.

Typically, with lists heavily based on citation metrics, there is also a risk that you may encounter authors or researchers that have artificially inflated their citation count, or that have lingering concerns over the veracity of various papers. The reason is simple; citations can be gamed.

For instance, a group of researchers who have all worked together in a small, niche field may be required to review each others papers. If they don’t flag the conflict of interest, then that paper sails through peer review, raising questions about how robust it was. More coordinated efforts are known as “citation cartels” or citation farming. Some journal editors have even been caught asking for scientists to cite their papers…

Look, citations can be fine as a measure, but really they are problematic as a measure. This is a known problem in science and we’ve done little to change course. What’s interesting though, is given there are 250 researchers here, and many will be something of a “top cited” researcher, there’s a chance they might also be involved in unethical practices that breach integrity standards.

So: I looked.

I’m not going to blow up the Australian’s Research Magazine in this newsletter,4 but I spent last week assessing some of the names on the list. I haven’t gone through the whole 250.

In the 96 profiles I did assess, there were data and publication concerns already flagged (on publicly accessible websites like PubPeer and RetractionWatch) in 10 cases — more than 10% of this small sample.5

So, did the Australian’s rankings take these concerns into account? Did they ensure the research this ranking is based on is… sound? How is its methodology able to recognise and account for this?

In at least one instance in the Top 250, I quickly found multiple retractions (where a scientist’s work is struck from the record because of concerns over data or integrity) for one researcher. These retractions were associated with the Wiley / Hindawi scandal of 2024, which saw retraction of some 11,000 papers because of “compromised” articles.

In this scandal, the scientific publication process was hijacked by bad actors who would often fabricate data or generate nonsense scientific papers, or place papers in special issues of journals where there was less editorial oversight, or take a fee and add authors who had nothing to do with the research to artificially pump up publication and citation counts. Basically it was a way for scientists to quickly inflate their scientific capital.

That’s the core problem that underlies the Australian’s Top 250. Using citation data, plucked from Google Scholar, as a metric for scientific excellence sends Australian researchers — and the public — the wrong message. The metrics we’ve used for so long are not just compromised but at the practically-broken stage. I’m surprised that some deranged-werewolf-in-journalists-clothing hasn’t pointed to this Top 250 yet and said “Well, who cares if CSIRO is cutting jobs?! Its research isn’t even that important?!”6

The problem with rankings like this, and how they might be manipulated, was marvellously outlined so 50 years ago, by Steven Kerr. It’s the idea of “goal displacement”. Here, the Australian intends to reward the researchers publishing papers in the best journals and getting the most citations. This will certainly capture some of those people.

But it will also promote the idea that if you want to be a Top Researcher, you’ve got to get good papers and lots of citations. This, we’ve seen, sometimes ends up in people taking shortcuts. Now the goal isn’t do good science and make the list… it’s get citations and good papers. One may argue that this promotes good science. Sadly, we have seen it does not.

There is surely a better way to rank our researchers, if that’s even what we want to do.

Thanks for reading no breakthroughs. This post is public so share it, and get more science journalism happening…

And a note on the matter of funding a Research magazine…

The Research Magazine itself is 60 pages long. Of that, 1 page is a cover, 1 is a table of contents and 29 pages are dedicated to advertising. It includes six pieces of “partner content”, with UNSW, Deakin University, Victoria University, the University of Southern Queensland, Swinburne University and the new Adelaide University getting a chance to spruik the advantages of their uni. It’s unclear to me whether or not these universities paid for this content and associated advertisements that fill the pages — one would assume they did!7

A cursory glance shows that NewsCorp made some $8.45 billion in revenue in 2025, and $648 million in profit — a major increase from 2024. I am not privy to how universities decide what to fund, but I will say that when I have come to several Australian universities with my hat in hand, pleading for just a couple thousand dollars to support independent science journalism via the Science Journalists Association of Australia… I don’t hear any replies.

Maybe I’m not going to the right people — if those people are reading this and want to support a vibrant, sustainable science journalism landscape then hey!! I am here!! — or maybe there’s just a real lack of imagination about where we could put our dollars and what that money could support in Australia.

Because if there’s one thing the CSIRO cuts has shown, it’s that big publications are very quick to write about the sad state of science funding in Australia right as it reaches crisis point. It’s always about the cuts. We get the analysis from political journalists, ready to wax lyrical about why these cuts are so terrible. But where are the science journalists? The ABC has a few, mostly on radio, and there’s excellent reporters in the Nine newspapers.

A science journalist would have flagged that the citation and impact score metric is flimsy, at best, and potentially a dangerous, misguided metric at worst. So, I really don’t want to yuck anyone’s yums here. If you made this list, great, and hopefully you’ll continue to do great work. But all week I saw universities posting stories about how they featured in the Top 250, or their researchers did and yet, again, no one recognised the real pressure that rankings puts on their academics, students and staff.

The Research magazine is a nice idea, and it highlights through some compelling profiles a few researchers that are doing great things. But, for the most part, it’s executed in a way that just doesn’t align with the modern scientific landscape, or the mountains of editorials and data collected on the chaos that “publish or perish” culture has unleashed. Hopefully by the next Research magazine there will be a science journalist on board8, and we’ll promote the idea that Australian science isn’t just about citations and top journals, but about what our researchers can achieve through curiosity, passion and, often, against the odds.