d i g i t a l   a d v i s o r . . .

Eight Important Features of Observation Research

You are in a funny place when you observe others in their natural setting.  The very fact that you are present means the setting is no longer their natural setting.  It is their natural setting plus you.  And they know why you are there and they are self conscious about your presence.  They will act differently because of you.  In addition, as a human being, you will have human reactions to the things people say and do that may be irrelevant to the study you are conducting, but that will nonetheless feel important and significant to you.  Here are some of the things you need to do/consider as you engage in observation research.

1. Your presence will influence on events

When you observe, you are a part of the situation, but you are not normally there.  What people say and do in your presence, therefore, may be different from what they would normally say or do.  For instance, if a classroom teacher knows you are interested in student engagement in mathematics she may be more likely to teach toward engagement when you are present than she normally would.

2. The first things you see will be most memorable

Psychologists call this the Primacy effect. Your initial impressions of a scene or person can have a distorting effect on later judgments.  Stick around long enough to get over these.

3.  You will have an emotional response

.  .  .  to the person or people you are observing.  If your initial response is to like or dislike a teacher you are observing, you will be inclined to judge all other aspects of that person's performance either positively or negatively.  So, for instance, a teacher who is caring is perceived to also be effective in teaching mathematics.  Don't fall for that inference.  Sometimes people you hate are actually pretty good at what they do and kids may like them too.

4. You will exaggerate the importance  of unique events 

Often observers are impressed with particular moments they observe-- student insights, perhaps, or clever teacher moves.  These moments are special, and they are real, but they may not be typical or representative of what normally occurs in that classroom.

5. You cannot assume the events you observe are similar to those you don't see

You may observe, say, a reading lesson or a math lesson, and assume that what you see typifies how things work throughout the day.  Or you may schedule all your observations on Monday, not realizing that students behave differently on Mondays than they do on other days of the week.  To be sure your interpretation of events makes sense, you need to sample events across different times, subjects, schools, teachers, or whatever dimensions are relevant to your study.

6. You will succumb to confirmation bias

One of the most difficult aspects of human nature is that we tend to seek only evidence that confirms our own prior ideas.  This is called a confirmation bias.  The secret to good observational research is to seek out counter-examples.  If you expect, for instance, teachers to take a custodial point of view, make an effort to find instances that defy this expectation, so that you can learn more about when they do and when they don't behave in certain ways.  If you expect them to represent a subject with a particular bias, search for examples of an opposing bias. 

7. You are always missing something important 

Understanding classroom dynamics requires you to follow the main story line.  However, it is easy to become distracted and miss important cues.  For instance, you may be focusing on one student while the teacher is focusing on another one, so that you will likely misconstrue the teachers' behavior or intentions.  Or events may happen very rapidly, and you will miss an important detail.

8. You will  change your own focus over time

As you spend more time observing, you will begin to see things differently than you did at first.  You may attend less to events that you consider routine, and more to unusual events.  Or you may become interested in new questions.  When this happens you will discover that your early observation notes are inconsistent with your later notes, making it difficult for treat them equally during analysis.   Plan on such changes.  Devise a method for tracking your own thinking.  One strategy frequently used is to write memos to yourself, or to write new hypotheses regularly.  Be sure to date these, because later, when you try to analyze your data, you may realize that all the notes you took before a certain revelation won't contain evidence about that idea because you hadn't thought to look for evidence about that yet.  Or, conversely, perhaps those earlier notes will provide a test for a hypothesis that you didn't think of until later on.

For help on how to manage observations and make them as fair as possible take a look at my page on controlling your Self and also my page on labeling the things you see and tips on observing.


© Mary Kennedy, 2006


Evaluating Research

Reasoning with Evidence

Doing Your Own Research