To Snare A Shonk

I recently received a tip about a significant number of papers by a single author in the field of social work.

In the last three weeks, I’ve been looking into it from my lil ol’ desk in the lil ol’ spare room of my apartment. Investigating these tips is not easy, or all that lucrative, for a freelancer. For one, they often lead to dead ends or walls that are impossible to climb. Two, even if you do find something shonky1, it’s unlikely there are publications who will want to publish that work, particularly in Australia.2

But this work… I love it man — this is the shit AI cannot do (right now (dear god let it hold)). It’s the kind of work that requires time and human oversight, that requires pulling red threads together on your crazy board, finding patterns, examining motivations. I wanted to give readers a little insight into how I go about investigating a set of claims, as a freelance science journalist.

The tip off pointed me to a researcher who, for more than a decade, has been publishing extensively, with ‘randomised’ trials and quasi-experiments that attempt to examine how interventions like yoga, faith-based initiatives, mindfulness, meditation and dance can affect all sorts of people from all sorts of walks of life.3

But the question is: Are the studies reliable?4

Investigationeering

Okay, so, the tip off comes in. Via email.

In this case, it doesn’t come from a Whistleblower, per se, and I don’t think I would call them a Complainant, either. Lets just call them The Concerned.

The Concerned had been doing some research, looking into social work studies and experiments. They had stumbled upon a series of single author papers that they found unusual. Let’s call the writer of these unusual papers, The Author.

The Concerned suggested, in their original email, that they were not making any assumptions except that, if these studies are real, then The Author deserves to be showered with rewards and praise. The work is stunning in its breadth and complexity.

I (Let’s call me The Chronically Underfunded Journalist Guy. No, maybe. The Spy! [I’m kidding, I don’t need a superhero name {could be cool though???}]), responded to The Concerned that I would like to speak and discuss the claims they were making. That’s Step One: Conversation.

After a Zoom call, they brought me up to speed. It certainly seemed like there was something funky going on that would merit a deeper look. The Concerned provided a list of several papers that might hold inconsistencies. I started to look into them.

The papers have a few unusual signals:

1. The studies document experiments featuring thousands of participants across multiple countries or continents. These people are found in places like schools, clubs and refugee camps. The Author states they often use spirituality organisations to find study participants. Some are related to the Brahma Kumaris, a women-led spiritual organisation out of India. These do have locations all across the world.

2. The ethics statements in these papers seem somewhat generic, and at times are exactly as the template provided by the journal. They also rarely feature ethics approval numbers5 and often the approval comes not from the researcher’s institute, but other institutes in the same city that The Author is seemingly not affiliated with. This is a strange signal. I haven’t seen that before!

There are two approaches here. First is to assess a series of papers and see if there is a pattern. I looked at a dozen of The Author’s papers. Both Fact #1 and Fact #2 hold across them.

3. The research typically features multiple sites. For instance, one study sees participants from Mumbai, Jakarta, Johannesburg, Pretoria, Harare and Singapore. Another? Multiple nations including South Africa, Kenya, Ireland, Greece and Germany. These are typically identified via “contacts and referrals”… Given the associations with spirituality organisations, this seems possible, and may well be possible. But the amount of work still seems extreme.
4. The way in which analyses were performed, tables were constructed and control and intervention groups were obtained all repeated, over and over. Often, the interventions resulted in some sort of positive outcomes over the test group — almost every study reports improvements in some way.

As I moved through The Author’s studies, these patterns were repeating. My instincts here were that there was something weird going on, so I decided to focus on one study and pull it apart.

A closer look

There is a single paper that is quite impressive, one that explored the mental health of refugees. In this study, more than four and a half thousand refugees across dozens of camps in nine different countries were part of an experiment assessing the effect of a spiritual education programme on their mental health and wellbeing. The group was examined before the intervention with a series of tests including trauma screening questionnaires and a mental health inventory. Then, after the six week programme, they were assessed again.

These people would likely be in a highly vulnerable situation — so the ethics approvals I would hope are rigorous.

In this case, the ethics statement says: “All procedures performed in the study were in accordance with the ethical standards of the National Association of Professional Social Work6 in India”.

That is strange: In accordance with the standards is one thing, and I hope they would be. But has anyone approved the study and methods? Do they need to? According to the paper, “the BSES MG Hospital and Research Centre, India”, did just that. I could not find a review or ethics approval from this location.7

With all this information, I emailed The Author.

I introduced myself and explained I was trying to understand the data in their refugee paper and that I wanted to write about their results, and that I was unsure how other scientists could replicate these results as provided in the literature. In addition, I asked about the ethics approvals and also asked if I could speak to colleagues The Author had worked with to obtain this data.

The response I received was that the ethics approval came in “2011-2013” by a special committee at the BSES MG hospital. The Author also relayed that a lot of this work was helped by the many faith-based organisations across the world, particularly associated with the Hindu communities and the diaspora.

I commend The Author for having such strong global connections. On the surface, that sounds pretty reasonable but I would still imagine there would be region- or location-specific ethics approvals required. I can imagine that it is difficult to work on this type of research and get funded for it, especially in India.

Upon further questioning, The Author mentioned that members of spiritual organisations helped with this research on the condition of anonymity and so could not put me in touch with them. Unfortunately, they also mentioned that the two members who had vetted ethics “superannuated long ago and succumbed to ill health during Covid”. Other collaborators I asked to speak did not speak English8.

After further conversation with The Author, I was provided a raw data file — an .xls spreadsheet. The file was named “prelim data”. I was honestly surprised by this! I would very much appreciate if scientists always provided the raw data when asked, but that hasn’t been my experience :|

Here is a screenshot of the first 30 entries of that data.

I have colour-coded the entries:

When I asked the researcher about this data, they told me “Sr No” stands for serial number. They also explained to me that the serial numbers were provided and input into the file in the order that “forms were received”.

Which means, according to the raw data we have quite a stunning coincidence. Not only did every form received alternate between male and female, but the age and education of every participant on the list appears to also repeat with an interesting cadence!

All the female refugees in these camps are also self-employed or homemakers!

If you’re 50 and above here? No formal education, but 30-39 and you’ve likely gone to university!

Until receiving this file, I conceded there were potentially ways a solo author could deliver this, particularly working with spiritual groups across different denominations. And, perhaps, there still are ways… I cannot come to a Whole Truth here without accessing all the raw data. The raw data is key. This data should always be accessible by those who want to view it alongside the publication, as long as all the relevant ethics and copyrights and all that stuff gets taken care of.

Perhaps The Author has conducted all this work, perhaps they have not: I can only get so close to the Whole Truth by looking at the publicly available information, plus the information received from The Author. Publishers should now investigate exactly how this data was obtained and if there are any other inconsistencies, including checking ethics approvals. Perhaps The Author will be able to provide the ethics approvals, or perhaps show that ethics approvals are not needed for intervention studies on refugees or children with autism or the deaf. I sure hope so.

As I said in the intro, I reported the issues across multiple papers to several journal publishers, including Springer Nature and Taylor & Francis. Both noted investigations into the work are now taking place. In fact, T&F have placed some flags on the work:

Great!

I think science journalists have a responsibility to inform publishers of inconsistencies in the scientific record if they are spotted, but I also think that researchers who have concerns can do this, too. Use PubPeer, ask questions, help with post-publication peer review. This strengthens the literature!

This particular case may well turn out to be backed by excellent data, ethics approvals galore and, if so, The Author will be rightly commended and applauded.

But…

The problem is if this data is somehow fabricated then … it’s the exact type of study we could expect to see polluting scientific literature in the AI era.9 The literature is in danger.

Some of The Author’s studies stretch back to 2016, 2017… It was a little more time-consuming to do this back then. Now? Whole datasets can be constructed in the blink of an eye. We’re sleepwalking out an open window and into a diabolical pit of despair. There’s no reason why studies like this can’t be quickly generated by AI, full tables of made up numbers that are then able to be analysed with basic statistics tests10 and sent on to publishers.

This has become a major talking point over the last year, but specifically in the last few weeks. The Rolling Stone had a piece about AI inventing citations just last week. Low quality papers are surging. The problem is way, way bigger than that and strikes at the heart of incentive systems and how we adequately review science in this, the Years of AI.

In 2026, I suspect there will be a reckoning of just how bad things look since genAI systems crept into our brains, how compromised peer review systems can be and what the requirements should be in regards to providing raw data. While some reviewers already ask for the raw data — and some publishers request it — we’re going to need far clearer guidelines to beat back nonsense in the literature and make sure scientific trust and integrity is maintained.

I’ll leave it there. More in 2026,

Enjoy your holidays.

Some scribbles

• I see that Crikey asked Australia’s ‘biggest media figures’ for their thoughts on AI and who is 'up-and-coming’. In the list of respondents, there are two cricket writers, two lawyers and zero science journalists. That’s a huge blind spot, given AI is upending science and science journalism will become incredibly important in informing the public of this scourge.
• In 2026, I hope to record all of the instances of contacting a research integrity unit at a publisher and assessing how long it takes to bring enquiries or investigations to conclusion. My contacts in the sleuthing and integrity world laugh at this, and I would be interested to see the data they have if they’re willing to share ;)
• You can send tips and leads to dctrjr [at] proton [dot] me.