d i g i t a l   a d v i s o r . . .

How to Control Your Self

You can't engage in any activity without using your own prior knowledge, assumptions and theories. These ideas help guide your actions.  But they also narrow your vision so that you see some things but not others, or see some things as significant and others as mundane, or assume you know why things are happening rather than questioning why they are happening. When this happens, you are unable be surprised by our findings. You will succumb to confirmation bias, the tendency to notice the things that confirm your beliefs and overlook the things that contradict them. So how can you use your evidence to challenge your prior ideas rather than to confirm them. Many of the "methods" you read about are designed to help you do this--the use of comparison groups, for instance, and the use of triangulation. These methods are intended to force you to look at things differently, to test your assumptions, to notice discrepancies.  Here are some tips. 

1.  Look for Gaps and Ambiguities in your favorite Theory

If you really are smitten by a particular point of view, try to find a some area within it that is less certain. Look for details that are not clearly understood, or for an internal contradiction in the theory, or for some as-yet unanswered questions that the theory raises. Focus your research on these areas, so that you can fill out the theory, add detail to it, or clarify some ambiguities in it. In this way, you can still be surprised by the outcome of your study even if the study rests well within that particular framework.

 2. Contrast two theories or frameworks rather than unilaterally adopting one

Instead of using your favorite conceptual framework to interpret what you see, pose a challenge to that framework. Compare at least two interpretations of one event. Or, within a given frame of reference, identify an area that is not clearly understood, an internal conflict in the theory, or an as-yet unanswered questions that the theory raises, so that you can still be surprised by the outcome of your study even if you are wedded to the particular framework.

3.  Compares two or more kinds of phenomena rather than just describing one

Your study will be much stronger if you compare two or more things rather than simply describing one thing.  Instead of studying how teachers respond to a particular new institutional policy, try comparing two institutions with different policies and see how people respond in each setting.  Instead of looking to see how students respond when you do "x", try also seeing how they respond when you do "y" and compare the differences. 

4. If not a comparison case, perhaps a comparative benchmark instead

Instead of asking a general question about how things are working, or what people are doing about "x", compare what they are currently doing with what they might have been doing otherwise. For instance, instead of asking, say, "How do teachers use technology?" ask how they use computers in comparison to textbooks or the overhead projector or the chalkboard. Instead of asking how students respond to a new curriculum unit or classroom activity, ask what would have happened in that classroom otherwise, and try to learn how students would have responded to that benchmark situation. Ask how they respond to this teacher in comparison to that teacher, or how they respond to the teacher's lecture in comparison to the teacher's group discussion, or how they respond to one assignment in comparison to another. 

5.  Use multiple working hypotheses   

Develop your question in such a way that it enables you to pursue and test multiple possible hypotheses (or conjectures or speculations or tentative ruminations) rather than focusing only on one.  The use of multiple hypotheses assures that you are not simply documenting everything that fits your favorite conceptual framework and ignoring everything else.  When doing this, the speculations have to each be plausible, you can't just develop three straw men and use them to further advance your favorite idea.

6. Look for Disconfirming Evidence as well as Confirming Evidence

When looking at your evidence, think about the things you see or hear that would support your ideas and then think about things that would not support your ideas. Then design your study so that you actively seek both kinds of evidence. The combination of confirming and Disconfirming evidence will force you to think about why both kinds of things occurred, to wonder what distinguishes the situations when each kind of thing occurred, and to learn. 

7. Imagine a world that works exactly opposite of what you think

Try to outline a world that works exactly opposite to the one you think you see, and then ask how you could tell whether that world or your world actually exists. For instance, if you believe dominating teachers discourage students from participating in class, imagine a world in which disinterested students force teachers to be dominating. How could you tell which world you were in, if both worlds consist of dominating teachers and passive students? What kind of evidence would you need to persuade someone from that world that your world actually exists and that theirs does not?Quantitative researchers call these opposite hypotheses "null hypotheses," but the same general idea applies to qualitative research. The idea is to posit a null hypothesis and then see if you can prove that it is wrong. 

8.  Imagine a world with multiple influences 

Imagine that the system you are studying is a complex machine, or an organism with many inter-dependent parts.  You are looking at how just one part connects to another.   You have settled on one factor as the ultimate cause of social problems, or you have settled on one solution as the ultimate panacea for your designated social problem.  Make a list of all the other contextual variables that might have an influence on what you are studying, and describe how these other factors might moderate your theory. Then look for evidence about those. For instance,

  • If you believe that teacher expectations cause students to thrive or wither in the classroom, think about how students might influence teacher expectations, how parents might influence student expectations, or how parents might influence teacher expectations.  Odds are, a careful consideration of other extenuating circumstances will force you to tone down your own hypothesis.
  • Ask yourself how your findings would be different if you had studied a different teacher, a different school, or even the same teacher during a different year.


©Mary Kennedy, 2006


Evaluating Research

Reasoning with Evidence

Doing Your Own Research