d i g i t a l   a d v i s o r . . .

What makes a Research Question Answerable?



1.  It asks about the real world rather than a hypothetical

Most of the practical questions we ask involve wondering what we should do about something.  These practical questions rarely make good research questions.  A good research question is one that can be answered by looking at the existing world: to things as they currently are, rather than to things as they might be, could be or should be.  Instead of asking, say, what an institution's policies should be on an issue, ask how people are responding to the existing policy or ask what their attitudes are toward a proposed policy. These are questions that can be answered in the real world, by observing or interviewing real people in real time.

 

Convert questions like this:
Into questions like this:
How should we teach division by fractions? 
 
What is the difference in student responses to method A and method B for teaching division by fractions?
 
What should teachers do when students come in late?
What do teachers actually do when students come in late, and what is their opinion of the benefits and drawbacks of these approaches?
 
What can I do to improve my approach to teaching about the Holocaust?
How will students respond if I alter my unit in a specific way?


The questions on the left ask what could be done or what should be done, while the questions on the right ask how things are actually being done and how satisfactory these methods are.

2.  You have real-world definitions of all the terms

Most of the things educators are interested in are broad concepts-things like children's understanding of mathematics, college students' attitude toward science, or teachers' beliefs about student diversity.  These terms need to be defined clearly enough that (a) you will be able to tell when you see them in the real world and (b) others will know what you mean by them.  What will you count as evidence of "understanding" or of "not understanding?"  What will you count as evidence of a teacher belief?   Some people recommend what they call "operational definitions," meaning that you are defining the terms specifically for the purposes of your own study.  For instance, you can say, "for the purposes of this study, the term 'understanding' means 'being able to respond successfully to the following questions or problems: . . .'"

Convert questions like this:
Into questions like these:
If I present the American revolution as a case of post-colonialism, will I encounter emotional resistance from students?   
 
Will there be more or fewer voluntary comments during class discussion?  
 
Will there be more or fewer arguments among students themselves?
Will I encounter more or less interest in the topic?
 
Will students do more independent reading on this topic?
 

The questions in the left column use terms that are not very well defined and leave you open to interpreting whatever happens as positive or negative evidence. The questions in the right-hand column translate these terms  into a set of specific kinds of outcomes that the researcher might be interested in.  You may disagree with the specific I offer in the second column, and if you do, you proving why it is important to define what YOU mean by these terms rather than letting everyone think they know what you mean.

3.  It has a manageable set of possible answers

You want to avoid questions that are so open-ended that the list of possible answers never ends.  For instance, the question, "What will happen if I do "x" in my classroom?" could have any number of answers.  Some answer might have to do with how students respond, some might have to do with the changes in lighting, temperature, or cleanliness of the room, and some might have to do with what parents or other teachers will think.  You need to focus your question on a particular list of possibilities that you think are important.  Instead of asking a broad question like what will happen if, ask whether students are more likely to do a, b, or c when you do x.    Instead of asking "Why does x occur?" ask whether x seems due to reason a, b, or c, which you have reason to believe are important possible influences.  Instead of asking, "how do teachers handle discipline problems," focus on a specific type of discipline problem and a list of specific ways of handling them that you think differ in important ways.

Convert questions like this:
Into questions like this:
How do teachers treat evolution? 
 
Do they focus on the natural selection aspect of it?
Do they focus on the randomness of it?
Do they focus on the timeline of it?
Do they address between-species change or only within-species change?
 
Do they mention intelligent design at all?  Do they describe it and then dismiss it?  Do they explain the difference between ID and evolution?
 
What would happen if I alter my approach to teaching fractions?
How much more classroom discussion will I generate? 
How many more students will pass the quarterly exam?
How much time will it take?
 
Why do students drop out of school?
Which of the following emotions are most prominent among drop-outs: feeling of alienation, or feeling of inadequacy?
 
Which of the following stimuli are most frequently mentioned as motivators by dropouts:  Peer pressure or bad experiences with particular teachers?
 

The questions on the left all allow an infinite number of possible answers.  Those on the right clarify  what the research is actually interested in. 4.  You cannot answer it with "yes" or "no"  Though you want a manageable number of possible answers to your question, you don't want the question to be so streamlined that it can be answered with a simple "yes" or "no."   You don't want to ask, "If I do "x", do the students do "a?"  Such a question begs a host of related issues such as how often students do a, how many students do a, and whether students are also doing b, c, or d, which might ultimately be equally or more important.

Convert questions like this:
Into questions like this:
Is Mr Jones a good teacher? 
 
How do students respond to Jones during free time?
How do students respond to Jones during mathematics lessons?
How do students respond to Jones during reading?
How to low-ac
 
How do low-achieving students respond to Jones?
How do high-achieving students respond to Jones?
Is this a good curriculum?
Is the teacher able to implement this curriculum?
Can students follow the sequence?
Are recommended exercises feasible?
Are students intellectually engaged?
Are students emotionally engaged?
 

The question on the left can be answered with either yes or no, but they don't allow the researcher to give important details about what has actually transpired.  Those on the right provide more opportunity for detail.

 

©Mary Kennedy, 2006

 


Evaluating Research

Reasoning with Evidence

Doing Your Own Research