Publish or Perish - part 5: Fishing for data

This going to be the last entry for the Publish or Perish blog series…at least for now. I wanted to end off by mentioning a little trick that some researchers like to use to get their studies published. Before I mention that trick though, let me tell you how research is supposed to work. In real research, someone forms a hypothesis (educated guess) about a particular area of interest and then designs a research study to either confirm of disconfirm that hypothesis. For example, let's say a researcher believed that eating pizza for dinner every night will cause cholesterol to be higher compared to people who ate salad for dinner every night. In this case, the researcher would run a study under controlled conditions in which there was a pizza group and a salad group. The researcher would collect cholesterol levels before or after the study and the hypothesis would be confirmed or disconfirmed.

In real life, researchers do not always find the relationships they expect to find. Some will try to publish unexpected findings but others will not even attempt to do this because they assume the editors will not find the results interesting. There are exceptions, but generally, researchers try to publish studies where there are statistically significant findings. In the quest for statistical significance, some researchers collect a bunch of assorted data points and try to find some type of statistically significant relationship between them. This is known as "fishing for data" and there is nothing inherently wrong with it as long as one admits that this is what they are doing. The problem is that many people do not admit it, find some type of significant relationship between two variables (for example, cholesterol levels decrease after drinking a certain juice) and then act like they had hypothesized this all along. They are working backwards rather than forwards but it makes the paper look better and increases the chance of getting it accepted for publication. In the end, this hurts science because the authors are not being honest in how they obtained the data. For example, the more analyses conducted, the more likely that some findings will be detected by chance and do not actually exist. So if you are going to go fishing, please be honest about it and don't pretend that you are doing hypothesis generated research.