You can make the data say almost anything. IMO, this happens way more than anyone thinks. When you only have 2-5 people that _actually_ read a paper and they have to slog through it, its not too shocking stuff like this could happen.
In my experimental design class in college, I remember the professor talking about the difficulties of dealing with data and what to include and what not. He pointed to a case where a datapoint looks like an anomaly and possibly should be removed. He showed the math behind it and including it means the experiment doesn't show a positive result and excluding it does. So which to do you do? If including it means you don't get funding for you and your team, what do you decide? This, of course, led into the ethics portion of the course, and how easy it is to go down a bad path, because you can manipulate data to make it say what you want.
You're describing a legitimately hard problem faced by honest researchers. This case, it seems like we have enough evidence to suggest that we are not dealing with honest researchers, but rather deliberate fraud.
That's not a hard problem for an honest researcher. Just explain the risk of that data point in the grant application and if the funders decide not to take the risk, that's their perogative.
Long while back, I had a big pile of numbers and I knew they could offer some meaning if only I could extract it. There's this whole discipline that advertises techniques for doing that, called "Statistics", so I looked there for lessons.
What I found was "How to throw away data that doesn't support your desired conclusions," for the most part. "Actuarial Science," a different field, had some useful techniques but not many. They're most interested in ensuring the bad data doesn't get into the tables in the first place; but at least they are doing "data on data" comparisons and not "data to expectations"
We're building "AI" right now but think about the inputs those see: The very first step is to throw away the statistically too common "stop words" ...
> What I found was "How to throw away data that doesn't support your desired conclusions," for the most part.
What exactly are you referring to here? This seems like a wildly misguided characterization of statistics, which I am sure cannot be based in expertise or practical applied experience.
> We're building "AI" right now but think about the inputs those see: The very first step is to throw away the statistically too common "stop words"
This is a fundamental misunderstanding of what a "stopword" is and how it's used.
Words like "the" are hard to utilize within with a bag-of-words model specifically. Removing them is not something people do/did because they are clueless monkeys. The goal is to improve the signal-to-noise ratio.
For example, traditionally spam filtering uses a very crude variety of bag-of-words model called "Naive Bayes", in which we assume (wrongly of course) that word choice is completely random, and that the only difference between spam and not spam is that random distribution of words. Are you really going to argue that the word "the" is critical to that process? If you can build a better NB spam filter by including stop words, by all means go ahead and do it. But both linguistics and decades of success in the field are against you.
On the other hand, words with grammatical function like "the" are absolutely important and relevant to the overall structure and meaning of a document. Therefore, training pipelines for modern deep-learning-based LLMs like GPT don't remove stop words (as far as I know at least), because the whole idea of a stopword doesn't make sense in a model like that.
I want to be respectful here, but it sounds like you took a cursory look through three vast literatures, without the perspective of having actually used any of this stuff in real life, and drew some invalid conclusions.
You're entitled to your own opinion of course, but your conclusions appear to be based on beginner-level misunderstandings. That doesn't seem like a constructive or productive way to conduct oneself through life.
There are so many complications because of data fraud and the fear of perception of data fraud.
I'm the guy who builds the experiments on a team of user researchers. There are all sorts of things that seem intuitive to an outsider but are poo-pooed by practitioners as unethical. For instance, you might run a study that doesn't have enough participants to have a statistically significant conclusion. An outsider would deploy it to more participants to see if the trend becomes significant with more data. A trained researcher will cringe at that proposal.
So far as I can tell, researchers consider the experiment final as soon as you peek at the data. If you want any changes - more data, different demographics, etc - you have to throw out everything and start over. Even though it's logically interchangeable, the data you've already collected is considered spoiled, because they don't want allegations of tampering/data grooming.
>"When you only have 2-5 people that _actually_ read a paper"
Even when a paper goes through a vigilant, rigorous peer review, it relies upon the data that the research team supplied. Over and over again cases like this have encountered manipulated data. Humans are flawed creatures, and if you spent a lot of time and professional credibility on a hypothesis, there is a strong motive to find what you sought. Doubly so if the results are salacious or contrarian in some way and thus get viral attention. Just convince yourself that it's the data that was wrong somehow and you know your assumptions are right so just this One Time you need to do a little manipulation.
A study I have seen cited on here countless times is the "honesty pledge" one by Ariely et al. It was the one that claimed that when a person signs a form at the beginning, they're more honest. It was complete and utter BS, based entirely on fabricated data. It joins an infamous list of studies that have had enormous influence (especially if they have an "aha!" factor -- if it is the sort of thing that Malcolm Gladwell would talk about, consider it suspect) but were the creation of someone making up data in Excel.
If I'm not mistaken, Ariely is asked about this on the "Armchair Expert" podcast. His claim was that he did not manipulate the data personally and it was someone further upstream whom he trusted. His point was that at some point, trust has to enter the process, except in the rare cases where a single person is doing all the research themselves (apologies if I'm misremembering this, but I think the following point still stands.)
IMO the researcher still has some responsibility because ultimately, it's their research. So the questions to me are:
1) How much due diligence is reasonable? Does it change depending on the source? For example, is it more/less reasonable to accept government-provided data at face value vs. data collected by an undergraduate?
2) What processes can be implemented to safeguard data manipulation? I know there is a movement to provide data with peer-reviewed submittals, but it's still a low probability that a peer-reviewer has the time or inclination to really dive into the data to assess the claims.
Ariely tried to frame both his assistant and the insurance company he got the data from. The insurers provided the same data to a journalist who found there had been very substantial alterations made. The assistant showed that the Excel metadata indicated Ariely was the last to edit it.
Yeah, I read the same digging a bit more after I made that original comment. It’s much more damning than what he made it sound like in the interview, although it didn’t say how substantial his edits were.
Very substantial, to the extent that the dataset was made up. The original data obtained from the insurer showed no effect.
The more interesting thing though is why he chose to investigate this question in the first place and why he chose to do fraud to make it seem true. The hypothesis is a very weird one and there's no reason to think it would hold. Unless that is you think of people as being child-like lumps of Playdough, so easily manipulated that trivialities like where exactly something appears on a form can yield huge behavioural differences.
That belief is the only reason you'd ever come up with such a hypothesis, and I think it's not really surprising that someone like that would engage in fraud. After all they have spent months (or years?) on trying to prove that people's levels of honesty are trivially controlled by psychologists like yourself. If you believe that's true then why wouldn't you commit fraud? After all you can easily manipulate people into not noticing it.
The manipulation seems substantial, but the point I was alluding to was that there wasn’t a smoking gun (at least by the amounts that I’ve heard and read) that Ariely made those substantial changes. I’m not sure what was included in Excel metadata, but it’s at least conceivable that the copy/paste + random change edits were done by someone else prior to Ariely edits. Where it gets damning is that he was the one the original dataset was sent to and the last to edit it. At the very least, it shows a lack of due diligence in not catching there were somehow many more datapoints added.
Just conjecture of course, but this came at a time when governmental “nudges” were very en vogue. I could see where successful research could be thought of AAA a pathway to influence, prestige, and money through government grants and appointments. And there were some highly regarded behavioral psychologists who were substantiating its effectiveness.
I think if the data goes insurer -> Ariely -> assistant, and both insurer and assistant present evidence that they didn't do it, then there is certainly a suspiciously large amount of smoke for there to be no fire. Short of CCTV footage showing him doing it, it's hard to get stronger evidence.
Yeah, governments love the idea that they can influence the population via simple tricks. That's understandable.
Unfortunately nudges are still very much en vogue. COVID was nothing but endless nudging, maybe more like pushing, with tricks like making everything into a social responsibility towards others being deployed endlessly even when not supported by the underlying facts. It worked extremely well. That said, I'm not sure you need psychologists to tell you that "do it for your grandmother" is a powerful manipulation tactic. A lot of the valid findings in psychology are obvious, and the non-obvious findings are often invalid. So we could just defund that field and not lose much IMHO. I say that as someone who has studied psychology. I have a good friend with a PhD in it who thinks the same.
Oh for sure. It would be very easy to determine if the data is bad (in most cases) if someone tells you where to look. It is almost impossible to expect people to put their mental resources to trying to "crack" every paper that comes across their desk tho... so we have a bit of a problem, to say the least.
Yep. I actually am a co-author of a paper that has been published and while I was working on it, I realized how easy it would have been to make the data agree or disagree with the hypothesis. Based on the feedback of reviewers, you see they don't dig into how you get your data and it seems as if its based on "good faith."
I could have made a mistake, or I could have been malicious. I don't think they would have caught it because it would have involved hours and hours of work on their part.
With current complex data processing pipelines it is almost trivial to add e.g. a wrong sign to some variable to get "results" from data that doesn't contain any. I've had many "results" disappear after I found a bug. I could have probably gotten papers quite easily by ignoring the bug.
I'm quite certain a huge share of "results" are due to bugs. Probably many of my own too even though I stress about this constantly.
An intentional bug would be practically impossible to show to be intentional. With notebook/REPL style analysis there wouldn't necessarily even be any documentation of the bug. I'd wager it actually happens, and even surprisingly often. We only know of fabricators who are bad at fabrication.
Absolutely, but most of this stuff happens outside of what shows up in the paper like coming up with excuses on why data points can be dropped or shopping around for different statistical methods that make things look the best. There can even be legit reasons for doing this kind of thing and that's what makes it hard to detect, it's basically just the honor system on whether you're "p-value hacking" or following common methods.
This is kind of why it annoys me a bit when I hear people harping on about trusting science, most science is not as simple as finding objective truths and just reporting them. That's not to say all science is bs and you're better off consulting a magic 8 ball, just that it should never be discouraged to look at methods and conclusions with a critical attitude. There is room for things to be fudged or pushed and very strong incentives for people to do it given how much money and prestige are on the line. It doesn't even have to be as big as a drug trial, one high profile publication can be enough to make a career so you can see how tempting it can be to just change a couple pixels in an image to boost a theory you earnestly believe is true
"read" is a pretty loose term. I think its a house of cards. When you cite something, you do so (usually) because it supports your paper, basically "X did Y and we need Y to be true for the foundations of this paper." When you cite "X", you do so with the assumption that X did their due diligence and peer review would have caught any issues... but its still only an assumption. If you had to re-create every experiment for every paper you cite, I'm not sure if one would ever actually finish their own research.
I have only ever published one paper though, so take what I say with a grain of salt. It's just my experience.
With relatively new, relatively small citation counts, the numbers are probably indicative of the number of actual readers.
But well established, high citation number papers often take on a "shorthand" role. You'll often see them in introductory sections or other supporting text with statements like "previous authors have X", "common approaches such as Y", etc. Here they often have little to do with the core of the paper, they are providing context.
Now really people should have read them, but sometimes Jones, et. al. 1998 just becomes a collective shorthand for a set of ideas. As such people will quote it just because the papers they did read quoted it, etc.
Often, over time, a single paper becomes the landmark for a set of ideas, and just gets cited to pull those in by reference. In theory this is the paper that "invented" those ideas, but in reality it's more complicated. Overall it's not a terrible practice, as a way to frame things, but can be error prone.
If you read papers in the less reputable parts of science (public health, epidemiology...) you'll quickly notice citations going to papers that contradict the cited claim, or which are about something totally different, or that don't contain any support for the claim within them. It's common. Peer reviewers don't notice this stuff, journal editors don't either. They might scan read the paper but they aren't checking it adversarially.
There have been cases where an early citation had a typo in the title, journal name, date, or page number ... and then hundreds of later papers ditto the citation with the typo, indicating that they did not actually bother to get an actual copy of the paper.
