Consider this made-up story.
Pretending I am a successful pet store owner but want to get more people thinking about the benefits of pet ownership, I decide to fund a research project to investigate what kind of animal is best suited as a pet for us humans. I hire a couple of professionals who happened to be involved with pets and their owners. One of the investigators I hire for my study is a DVM (a veterinarian) named Bill. Bill runs a successful animal clinic on the edge of the city and speaks with many animal owners. Fred, the other investigator, is a professor of psychology at the local university. Fred teaches courses in pet therapy for folks with mental disorders. Bill and Fred agree to research what kind of pet was most likely to make a person happy.
Bill and Fred publish their research results in a local magazine. A local TV news channel picks up this human interest story and runs the following headline, “Secure your happiness: Own a cat!” For a few days, everybody is talking about the article. Sales in my store grow. I make more money than ever. I make more than enough to pay for Bill and Fred’s research. I start thinking I should fund another study…..
So. What do you think about the story the TV station ran? There was an article in the local paper, “Be happy; Own a cat”. If I wasn’t the successful pet store owner, I’d smell a rat. I’d be skeptical of the findings. Why? Read on.
In my story, I grew up in a home that allowed cats, lots of cats, but never dogs as pets. I own a pet store with all kinds of pets except dogs. No dogs are bought or sold in my successful non-dog pet store. In my store, I carry all of the non-dog pet accessories. I have a lot of happy non-dog-owning customers. I encourage all customers to write and express their happiness with their non-dog pet, perhaps relaying a funny or cute story about their relationship with their non-dog pet.
Bill, as it happened, developed a severe allergy to dog dander at an early age and suffered all through veterinary school, especially those days when he was working with dogs [and by the way, there are no hypoallergenic dogs (1)]. Bills animal clinic was 80% large animal, meaning that he mostly saw cattle and their ranch owners in his practice. Bill always asked his partner in the business to take the dog cases. Two months before I asked him to take on the research, Bill’s partner suffered a stroke and left Bill with a pile of debt. He said he could do the research, but it wouldn’t be in-depth research. I offered to help Bill with the research. Fred, who taught the benefits of pet therapy, also published a book titled, “The Essential Cat: Lessons in pet therapy”. Bill’s publisher, Eastern Star, had published many books on cats and thought the study might help Fred sell more copies of his book. Bill and Fred decided that a great way to survey people about pet ownership was to find those who already owned a pet. Using my customer contact list to set up interviews with customers who had offered stories about their cats.
The publisher of Fred’s book also owned the magazine that printed the research on pet owners. Fred’s publisher, also owning the local TV station and local newspaper, decided what got published. The TV reporter didn’t read the article before he made the 30 second sound bite. He didn’t need to. All he had to do was say something nice about cats being the best pets for humans, mention the two authors of the study and where they worked, and show a clip of my pet store. That was a nice touch. After all, I was paying the tv station to run ads for my store.
Every key player in this story about pets, and the article they wrote, broke almost every reason to trust the media stories that sprang from it. This story is make-believe. Sadly, there are far too many real-life examples of the making and reporting of science that happen just like this make-believe story. Except it’s not make-believe. There is a set of governing rules and protocols for research. It’s called the scientific method. Why didn’t the reporter stop the players in the make-believe story and ask a few questions? The reporter was part of the problem; he worked under a set of rules that protected all the players and, if he wanted a livelihood, he had to do what he was told. Unfortunately, there are real-world examples far worse than this made-up story.
How Can You Spot Real Science?
Train yourself to ask questions about what you hear, read, and see. The trustworthiness of a report starts with researchers adhering to the principles and intent of the scientific method. Once you know and understand the parts of scientific research, you can begin to trust—or not—the articles you read and the news clips you see. Researchers in all fields benefit from a trust that comes as a natural consequence of following the rules.
First Some Definitions
I need to define some of the words I’ll use in this post. This is important because if you and I have the same definition, then the conversation is better understood. In other words, we can compare apples to apples and no one thinks it’s an orange. I reference the Merriam-Webster online dictionary (3). I’ve used Merriam-Webster since I entered nursing school.
Method:a discipline that deals with the principles and techniques of scientific inquiry.
Discipline: a field of study.
Scientific: conducted in the manner of science or according to results of an investigation by science;practicing or using thorough or systematic methods.
Science: the state of knowing;knowledge as distinguished from ignorance or misunderstanding.
Hypothesis:a tentative assumption made to draw out and test its logical or empirical consequences.
From these definitions, I can say that the scientific method is a strict, reusable, step-by-step guideline used to study the world around me that elevates my understanding.
I have another book for definitions that uses the Latin and Greek derivations of scientific words (Marchuck 1992) (4). Marchuk defined the scientific method this way: The logical, orderly arranged approach used to study nature; usually consisting of the following steps: information gathering-> hypothesis formulation -> prediction -> experimentation and analysis -> conclusion (explanation).
What are the steps for the Scientific method?
The steps that Marchuk proposes for scientific inquiry could serve any researcher studying the natural world. The most important thing he says is that the work be logical, orderly, and arranged. In other words, don’t put the cart before the horse; do each step correctly and in order. This is how researchers observe and record what they see. As citizen scientists, we should be aware of the steps and how we use them.
Information gathering
What do you see, hear and read? Has anyone else seen or written about this? How often is it happening? The start of any inquiry is finding out what appears to be going on. Learning to ask yourself questions is the very best beginning to understanding something. Be skeptical; ask why or how a lot. Note what you observe. Some example questions are: is there a cycle, what is the timing, is there a pattern, does location make a difference, how did this start, etc. The more questions you ask, the better will be your big question; the hypothesis.
Hypothesis Formation
Your big question (hypothesis) is what you propose to be an explanation for your observations. Your proposed explanation will drive your research. Your question must be specific. In the make-believe story, the hypothesis sounds like it might have been, “do cat owners like cats for pets?” If, while observing pet owners and non-pet owners, Bill and Fred thought the non-pet owners were a bit less happy, they might have asked a question, “Can owning a pet elevate a persons mood?”
Prediction
Once the hypothesis is formed, you can make a statement that would explain your observations. Bill and Fred might state, “Pet owners have less depression.” The interesting part of making a scientific statement; the goal of scientific experimentation is to DISPROVE your statement. There’s more scientific jargon abound this (search for null hypothesis) but it’s important to allow yourself the freedom to be open to anything you discover. A better statement for Bill and Fred is ‘owning a pet affects depression.’ Words matter. Now Bill and Fred can set up the research that may show that owning a pet does or does not affect depression.
Experimentation
Decisions now have to be made about how to test the prediction. In a later post, I’ll expand on this, but for now, let’s just say that the methodology (how the test design) will hopefully determine if the statement is false. Bill and Fred could: gather a random sample of people, divide them into two groups (one group owns a pet, the other group does not), record the demographics for everyone, give each person a previously tested questionnaire about depression (called a depression scale). After one month the pets are taken away from the pet owners, and pets are given to the non-owners. After one more month is allowed to pass, the depression scale is administered again.
Analysis
How you analyze something depends on your methodology. Each method generates data. Not all data is the same and some statistics can’t be applied to all data. In Bill and Fred’s original paper the data generated was observational—they looked at stories and letters from happy cat owners and possibly interviewed some cat owners. The quality of observational data is not high on the list of data types (5)(6).
Conclusion (explanation)
Admittedly, Bill and Fred’s choice for a methodology could have been better. You can imagine what the depression is going to be for someone who has a pet taken away. Also, how would it feel to have the care of an animal thrust upon you if you don’t want to own a pet? This is called a limitation to the methodology. A research paper that doesn’t list limitations should raise a red flag. There are always limitations. The conclusion is the statement about the prediction at the beginning of the research; was it disproved? Something to remember is that scientific research never offers proof. The more repeatable and generalizable the results, the more trustworthy the conclusion. But science is never finished.
As you can see, using the scientific method builds trust in the observations of researchers. You can also see that the scientific method opens researchers to making mistakes, intentional or not. Deciding if we believe what we see on tv, read in the paper, or hear from a friend starts with understanding the scientific method. Be skeptical.
References
(1) Do hypoallergenic cats and dogs exist?
(2) Scientific Method:
(3) Merriam-Webster: https://www.merriam-webster.com/
(4) A Life Science Lexicon:
(5) Variable types:
(6) Observational Field Research:
Image Credits:
The Scientific Method:
Scientific Methods and Variables:
Four data types:
Joke: https://bit.ly/31vXuYA
[this is an abbreviated blog post]
