Hello Joe, hope you’re having a lovely day 🙂
1) http://itsdefinitelycaela.wordpress.com/2011/11/23/what-makes-a-research-finding-important-hw-for-week-89/#comment-23
3) http://psud56.wordpress.com/2011/11/20/whats-the-better-perhaps-more-scientific-method-quantitative-or-qualitative/#comment-11
4) http://psucc6.wordpress.com/2011/10/21/how-do-politics-affect-the-science-of-psychology/#comment-30
Hello again…it’s been a while, here we go on yet another journey together….hold onto your hats chaps.
I believe that although not fully developed yet, in that qualitative methods are not fully solidified and accepted as part of science, they do not violate the scientific method.
Firstly, I think it is important to know what we mean when we say scientific method. Well according the the multiple examples I have laboriously read it is a 5 stage technique which uses (1) objective observations from the world, to (2) painstakingly form a hypothesis which is used to (3) make logical predictions that can be (4) tested empirically and (5) repeat until there is little discrepancy between the prediction and results. New observations or findings can lead to the creation of a new hypothesis and set of predictions. So the vital cog of science is kept ticking….tick….tock…snore.
So we now know what the scientific method it…BUT do qualitative methods violate this….well…..
Qualitative measurements are based upon observation and they generally require some type of numerical manipulation or scaling. They allow for an in-depth generation of data, often methods include Focus groups, interviews and observation. Now the question becomes, can these methods slot nicely into the scientific method, do they follow the 5 stage model I mentioned above. Let us see…..
Well, qualitative data fits nicely into the first, second and third step in the case of all 3 method types I have mentioned. They use observations from the world to focus on and create a hypothesis, which leads to predictions which formulate a research hypothesis. Therefore, qualitative methods are in keeping with scientific method thus far 😀
(On a side note surely qualitative observations are required to be able to produce the research hypotheses needed to start the quantitative research….this is just a separate thought I thought I would share…lucky you.)
Now, step 4, the empirical testing. Qualitative methods use direct observations and interviewing to gather the data. However the data is not always unbiased and sometimes subjective. Data gathered by questionnaires are open to many avenues of deception, such as social desirability, reactivity and other biases, participants can manipulate their data to deliberately ‘help’ or ‘harm’ the research. Bias from the transcription of data can also apply. However, this is lessoned by quality control implemented such as inter-rate reliability, therefore perhaps here qualitative methods could be lacking.
Finally, qualitative methods do not appear to violate the final step as each method can be interpreted and analysed and the researcher can reflect, alter and refine their hypothesis if need be. It can be seen whether or not the data supports the predictions, and if not alterations can be made. Qualitative research clearly allows for the ‘back and forth’ approach used by the scientific method, as researchers can now return to step two. The Grounded Theory to qualitative methods is a bottom up approach in that it requires the data before construct a theory. This is therefore helps to keep things objective, because researchers will not be sneakily trying to make their data fit a theory, but rather finding a theory that helps explain their data. Part of this theory, is that it is continuously built upon, constant collecting of data which leads to adaptations in the theory.
Well, this whistle stop visit to qualitative methods has drawn to a close, and now the time is to conclude. I think it can be seen that qualitative methods don’t appear to violate the scientific model, although the methods are open to bias, multiple avenues of quality control are implemented, and therefore I feel safe in the knowledge that there is no more risk of un-subjective results as there is with quantitative methods, therefore qualitative methods do not violate the scientific method 🙂 Having said all this, I thoroughly believe that qualitative data is best when use in partnership with quantitative data, allowing for an in-depth understanding of human behaviour and allow for more broad theories. Qualitative methods do not violate the scientific method but add to it, giving more rich data for analysis.
Now that’s over….here’s a video of ducks being blown over 🙂
mother duck: “shake it off ladies”
So why are reliability and validity important i hear you say. Well interesting you should ask. My instant response would be because they allow you to evaulate your research, finding out whether your test can be applied over and over obtaining similar results, and whether your experiment does in fact measure what you intended it to. Vital bits of information, necessary to keep the clock of scientific research ticking away.
Reliability is the extent to which when an experiment is rerun on the same person you obtain the same results. Now why is this important? Lets say for example you are testing a new pregnancy test, 50 women test it and according to the test 50% are infact pregnant…YAY! However, you rerun the test a few more times and now the pregnancy readings range from 15%-80% being pregnant. Is this test reliable? No, the test was used by the same women with varying degrees pregnancy readings. As for whether this is important, most definitely, if 9 months down the line a women who took the test, unexpectedly gives birth to a little bundle of joy, I’d say she’s going to be pretty annoyed. I mean these little arrivals take planning and preparing for!! Reliability gives your findings and experiment design strength, if it is consistent then other researchers may adopt it….maybe….depends if its valid (nicely linking my next paragraph in, seamless!)
Moving on, validity is the extent to which your study measures what it set out to measure. Same example, 50 women take a test which was designed to detect pregnancy by the measuring the circumference of tummies. You now realise, using this measure many males have qualified as pregnant…..interesting. You re-evelaute your study and realise your actually just measuring the size of the individual and not whether they’re harbouring a child in there. Is this important? Erm, I’d go ahead and say yes, this type of validity is Face Validity, it clearly doesn’t look like what you set out to measure. It is important because 1) you can clearly see that advertising this as a pregnancy test is not a good idea. 2) you can re-direct yourself, although not good for pregnancy detecting it is good to see how much your tummy expands having eaten your way through 10 pizzas….mmmmm. 3) being able to detect validity issues can stop you embarrassing yourself 😀 BONUS!
Face it....it's not valid!
It is important to note though that although both important they do not always co-exist. If research is reliable this does not necessarily equate to it being valid (I can sense another example…excitement!). You decide enough is enough and you’re going to bite the bullet and weigh yourself. You know you weigh around 9.5stone, give or take a pound or two. You get on the scales and the first time it says you weigh 8stone…YES! However you get on again and now you weigh 13stone, you continue this and get a wide variety of results, this is not reliable. You get on the next day, having consoled yourself with ben and jerry, and you now consistently weigh 7stone, this is now reliable but not valid, it clearly isn’t measuring what you wanted it to, your true weight. You then realise it’s because you’ve been leaning against the wall, the scale now consistently weighs you at around 9.5stone, the scales are now reliable and valid. SUCCESS. For further explanation of the lack of peace between reliable and validity please look at the below diagram, it’s pretty nifty, and I think explains the whole situation nicely.
Continuing, it is also important to note there are multiple types of validity….most of these i am not going to name, because i don’t want to. However one I am going to name/already have named is Face Validity. Sometimes, having validity is not beneficial to research, as is so with Face Validity. If a researcher is measuring a less popular construct, for example dis-honesty, then if the experiment looks like that is what it is measuring participants may adapt accordingly. This could alter the findings, in that the participants give misleading answers, so they don’t appear to be dis-honest people. Tricky.
I’m going to leave it there, please comment and we can get a lovely debate going and rack up our comments, enjoyable 😀
Now a word of warning, if you don’t want the following to happen, 1) ensure your experiment is valid, and 2) ensure your cat isn’t evil!!
COMMENT 1: http://anythingforadegree.wordpress.com/2011/10/07/do-you-need-statistics-to-understand-your-data/#comment-16
COMMENT 2: http://anythingforadegree.wordpress.com/2011/10/07/do-you-need-statistics-to-understand-your-data/#comment-20
COMMENT 3: http://psucfc.wordpress.com/2011/10/07/do-you-need-statistics-to-understand-your-data/
COMMENT 4: http://psyalo.wordpress.com/2011/10/05/do-you-need-statistics-to-understand-your-data/
COMMENT5: http://psud24.wordpress.com/2011/10/07/“do-you-need-statistics-to-understand-your-data”/#comment-8
COMMENT 6: http://psych31.wordpress.com/2011/10/05/do-you-need-statistics-to-understand-data/#comment-13
COMMENT 7: http://lmr92.wordpress.com/2011/10/06/do-you-need-statistics-to-understand-your-data/
do enjoy Joe.
The removal of outliers is something which baffles me, when is it ok to remove them, and when is it classified as cherry picking? It would appear to me, in some situations removal is acceptable, but there are boundaries, researchers can’t just go deleting data sets ‘en-masse’ because they got trigger happy. Firstly what is it that classifies an outlier? It appears to be results which are evidently different from the structure of the other data for example 4+ SDs different, whether that be because the participant has deliberately obstructed their results, or whether the participant didn’t understand what he or she was supposed to be doing. These are all things which can cause data to follow a different pattern and can therefore alter the outcome of the study. It is therefore acceptable to remove outliers in some situations, and the blogging begins (deep breath people).
The removal of potential outliers is necessary in some situations, this is to avoid drawing invalid conclusions. If an individual has been asked to answer a questionnaire on how little they enjoy blogging involving several questions on a 1 to 5 scale, and instead of giving a truthful response they simply hit 1, 1, 1, all the way down. This should be clearly evident to the researcher, and would give the researcher good cause to remove that data set, as the participant has not provided valid data. The participant has deliberately given invalid responses. It would be worth while for the researcher to include some questions which contradict each other, this would make participants who have done this more clear, for example one scale of 1 (not enjoyable) – 5 (very enjoyable), do you enjoy having to blog EVERY week? A question later could be (same scale) is blogging something you enjoy doing? Doing this would help prevent the removal of data which is perhaps genuine.
The above reason is also transferable to human error, if data has be wrongly collected or inputted wrong then it may be necessary to remove the data set. An example is you are asking the hourly wage of a plumber, you know the average wage is around £30/hr and the standard deviation is around £10. Perhaps participant 25 answers with £37,000. It is fair to assume that here the individual has mis-read the question. Then this data set must be removed, because it is not valid for the research.
However, in some situations the removal of outliers is a little bit naughty. For example, if you collect the IQ of university students, perhaps everyones IQ is around the national average of 100, however you quite rightly spot a individual who has scored a huge amount higher 130, this is 3SDs higher than the mean. What do you do? This data cannot be removed! *pause for dramatic effect* Why?!?! It has been collected in a valid way, and the individual has, just by chance, and IQ which is substantially higher. This is a problem which researchers face daily, chance has dealt them a rough hand, intense. However, in these cases it can sometimes be useful, the outlier can be retained and perhaps spark new research, and so the circle of life (the scientific model) continues nicely.
Poor little guy.
Having said this, on removal of an outlier it is important to view the effect this has had on your statistics, has this removal substantially changed the outcome and therefore your conclusion. Has it, or will it have caused a Type I or Type II error? It is important to do your analysis, both with the outlier and without it. With reference to the above point, it may be worth while running an analysis without the outlier in, and therefore seeing whether this chance occurrence had a dramatic effect on the result. It would then, as stated above, be cause to either re-run the research or spark new research.
An interesting study at the Yale School of Medicine found that a shocking 80% of clinical trials financially supported by drug makers reported positive findings, this however is in stark comparison with 50% of those carried out by independent academics, perhaps someone needs to be reading this blog, have people been ’tweeking’ their data? :O
It is clear then that the removal of outliers is tricky business, and I have only touched the surface of the area. I haven’t gone into the intense statistical ways of spotting outliers and how to deal with outliers which perhaps aren’t removable. However, it is clear that in some situations the removal of outliers is acceptable. It appears as a rule of thumb, that removal is ok when the data set is invalid for one reason or another (participant treachery / human error / when instrumental error has occurred). However it is cherry picking when the researcher removes data which has been fairly and validly obtained but doesn’t support the hypothesis. It appears though, that this kind of decision is much based upon the judgement of the researcher, if they believe the results are invalid for whatever reason then the removal of the data set should be ok. However there must be strong reasoning behind such a decision.
Now, this was quite intense, please relax while watching the below video. F.Y.I ‘Cancelling World of Warcraft account’ = Having to write a blog every week.
Below is an interesting article on outliers and how it has been found that some researchers have not been revealing ‘inconvient results’, tut tut tut.
http://www.smh.com.au/articles/2004/05/31/1085855500131.html
Regardless of how you choose to analyse your results, it is vital you’re able to understand what your data is telling you.
Running any kind of scientific research has the potential to create thousands of bits of pesky data, for example I may ask the question do students perform better on IQ tests when they are full of pizza, or full of beer, my sample may be 75 and for each person I need a baseline and their performance in both environments. Having a list of all the data is only one minor part of completing empirical scientific research, you have to be able to see if there was a difference. Without statistics it would be close to impossible to understand what story is being told by the data.
This can then lead to the production of graphs. Graphs are a way of viewing the data in a more pictorial format, without having to run a load of statistical jargon. As well as looking rather pretty, they can help to give your data a more clear structure. You can view the distribution and see clear interactions, perhaps at what point pizza becomes toxic to a student. They also allow you to plot all these bits of a data and see if you can view a ‘pattern’. In some cases this may be enough to understand what your data is telling you, for example if you’re simply asking is there a difference? Perhaps a graph would be enough for you to be able to say ‘yes, there is a difference in performance when students are full of beer to being full of pizza’. However, when you’re asking the magnitude of the difference, purely viewing a graph may not give the full story. Waffling on, part of research is to support or reject a hypothesis and in turn create new hypothesis and therefore new research and data, surely the next step from asking if there is a difference is to find what that difference is, and therefore gain a full understand of your data. It would seem then that statistics are required…..gutted 😦 worst news since steve jobs death.
Having said this, for some researchers instead of producing a scatter graph with all the data points on, it may be better to plot only the means of all the treatments, for example plotting treatments within different time periods, for example toxic effect of pizza 3weeks in, then 6 months in and finally 1 year. The finding of means is a basic form of statistics. So we would need to have a basic understanding of the subject.
Again though, many researchers use qualitative data Henwood, & Pidgeon (1992) actually suggest that qualitative data is more effective as it allows researchers to collect information which is contextually sensitive, persuasive, and relevant. Graphs can be useful to display qualitative data, observational studies may produce data which isn’t quantifiable. Studies have been conducted where only qualitative data has been collected and graphs used to describe the trend of the data (Bass, 1998). It would appear then that in some situations in order to understand you data statistics are not required.
We now live in a age where computers can do alot of the fiddly sums for us….YAY for computers. However, the computer is only useful if we understand what it is telling us. Without the knowledge of statistics we would never know that a p value less than 0.001 is very significant. Although a computer can do the sums for us, we have to be able to take those figures and make sense of them, put meaning to what they say. In essence we have to be able to explore the data, take into account all the different variables and understand our data. Without a knowledge of how altering sample size will alter your result and other basic statistics it will be extremely difficult to gain a full understanding of your data. I guess what I am saying is, is it is one thing to shove all your data into a computer and press go, it is something completely different to gain a overall view of what the data is actually telling you. Although perhaps statistics are not required to understand some forms of data, they are a tool which can aid many people in their research as well as developing their research, ergo in the majority of cases statistics are required to gain an all round understand of the data.
Well statistically speaking…..
This is a two fold argument, the first being the natural impulse to state the obvious….statistics are somewhat tedious!! The second and more in keeping with my wanting to actually obtain a degree is, statistics are useful 😦
If we are to believe that money actually does make the world go round, then I guess there are HUGE benefits to having a strong statistical background. I mean, if i ever have to get a job (fingers crossed for the big lottery win), then I want someone qualified to be calculating my tax, no way am I being stiffed out of my ‘hard earned’ money, I don’t want those sneaky tax men swindeling one pence more than they’re is supposed to. I guess then, for myself, it is important to know statistics, so I can check up on these people*.
(*http://www.dailymail.co.uk/news/article-1361448/Britons-pay-13billion-tax-year-errors-confusing-benefits-system.html).
Statistics are also useful in deciding whether to take a risk or not. What are the chances of me winning this outcome? Linking back to my previous example, my inevitable lottery win. Statistically I have a 1 in 13,983,816 (lottery website), according to statistics I should not be taking this risk, the chances of me winning are very unlikely. Perhaps if I save all my pennies and invest them in a stock which statically is likely to make me money I’m more likely to make my millions. Fully understanding and learning statistics would have helped my mother invest in ‘Google’ back in 2005 when they were £100 a share, now they’re over £500 a share (uk finance)…..well played mum. Clearly with respect to money there can be HUGE benefits with reference to knowing your statistics.
This is an extreme example, but being able to accurately ‘weigh-up’ and predict outcomes can be applied to multiple avenues of life, a less grand example would be reading reviews on amazon. I’m sure many of us read a review or two before jumping in and purchasing that much needed padlock (my latest purchase), I was going to buy a padlock with a 5 star review until I saw the star ranking was based on only 1 review. I then rescanned and decided on a lock which although only has a 4.5 star ranking, it was based on over 50 reviews. I took my chances and bought that one. Simple example, but I think I made the right decision…do you? The benefit here being I was able to use statistical knowledge to know that it is better to have a 4.5 ranking from 50 people, than a 5 ranking from 1.
Now I would hope some serious statistical analysis was done when the government decided to phase out the old incandescent light bulbs, in favour of the new energy saving ones. These light bulbs had better last for 10years at £2 a pop, or I’m stocking up on the 25p for 4 old bulbs. Apparently Wall Mart is set to save a wopping $6million using this new bulbs alone!! (Davis Fessler). Having a statistical background helps you know whether what you’re buying into or not is true, people would be more inclined to question the things presented to us as fact, example “100% of people say this toilet roll lasts longer *small print – 10 people asked*. I’m sure once I’ve finished my course, I will be able to wap out some suitable formula to decipher whether it is in my best interests to be buying the old bulbs or new ones.
With reference to psychology, (my current degree) the importance of statistics is immense. I mean, human behaviour is extremely variable at best. Being able to create a ‘bell curve’ of the extremes to norm is important to help those who perhaps fall in the extremes, otherwise we would never know (Herrnstien, 1994). As well as the large amount of statistical analysis done on medication for example to find out whether that paracetamol you took yesterday actually had a significant effect on reducing the pain of writing this blog. Statistics can be used to find 1 combination of drugs out of thousands which is better at attacking a set illness. Without statistics it would be near impossible to look a a list of raw data and make sense of it and we would never know that combining drug A7 with drug Z2 is effective in treating depression. Statistics help us learn and move forward in discoveries.
We can therefore see the use of statistics are wide and varied, they are used from ensuring the shampoo you’re using is actually better than the other top brand. They are behind the claims that a new drug has shown to significantly improve symptoms of schizophrenia. Statistics can simply help you make a better decision about which product is better, or whether the new spin advertisers have put on their product is actually true. Statistics are clearly a benefit and therefore having a strong background in them can both help you in later jobs as well as in day to day life, and therefore having a strong background in statistics means you can understand a large proportion of events in everyday life. BOOM, DONE!
Who Blogs Statistics?!?! 