• Teachers Matter (NYT)

It’s not “the system” or “the curriculum” or “the standardized tests” that we should be paying attention to, except insofar as they get in the way of good teachers trying to do their thing.

At the moment I’m more interested in lessons we can learn from video game design and take into more traditional, classroom-based instruction than I am in creating an actual video game that teaches physics. (The latter, however, is also a fascinating challenge to contemplate.)

In that vein, a definition by Bernard Suits (quoted in Jane McGonigal’s excellent book Reality is Broken) caught my attention:

Playing a game is the voluntary attempt to overcome unnecessary obstacles. (p.22)

The key word here is “voluntary”. McGonigal makes a case that if it isn’t voluntary, it isn’t a game, and many of the remarkable phenomena associated with game-playing disappear. The entire psychology changes.

Yes, attending university is in principle a voluntary choice, as is one’s major; but beyond that, we pretty much tell students what courses they must take and what they must do along the way to succeed, and keep them in line with grades and transcripts. Does that doom any attempt to make learning more deeply game-like?

What I’m getting at is that the very structure of our educational system frames learning activity as an externally-motivated, externally-directed, authority-laden series of tasks and assessments. I’m concerned that trying to embed a novel learning micro-environment — say, a gaming-inspired self-paced learning activity — into such a matrix could be doomed to failure, not because of the micro-environment’s worth but because of drastic dissonance with the matrix.

If I’m even more ambitious and try to construct an entire course as something analogous to a game, I still have to assign a grade at the end, and students know it.

Those of us who would like to experiment with gaming-inspired alternative paradigms and challenge some of our fundamental assumptions about what instruction should look like, and who don’t have the luxury of creating an entire parallel educational system to do our testing in, need to worry about such things.

“Playing a game is the voluntary attempt to overcome unnecessary obstacles.” — Bernard Suits, quoted in Jane McGonigal’s Reality is Broken

Is learning physics a game? Is doing physics a game? Does it depend on how obligated we feel to do any particular task? Is attending university voluntary (or compelled by social and/or economic considerations), and if so, does that make the whole endeavor a game? Taking any particular course may or may not be voluntary; doing homework, lab reports, etc. rarely is.

Why does this matter? Because in general, people like games, and often reach their best performance (think flow state) while playing games. Perhaps we ought to be learning from the game design industry.

Perhaps the human organism is well-adapted enough that if we can give learners the freedom to pursue their own learning, with an adequately rich and suitably organized and accessible array of resources, they would naturally find a highly optimal balance of these factors?

In other words, maybe people know how to learn better than we know how to teach, and we just have to figure out how to let them do it. And stop convincing them that learning something like Physics means doing it “the school-like way.”

Afterthought: I suspect we can’t do this because so much of the educational enterprise is designed to get learners to learn things they’re not particularly invested in learning. Maybe we need to revisit that?

• http://bit.ly/beatty-aapt-2010

Update: Here’s another link, in case bit.ly goes under:

• http://goo.gl/XPgM

An intriguing research result is that formative assessment may actually be counterproductive if the teacher doesn’t have adequate strategies for responding to that information. Here’s a quote about that from a paper by Dylan Wiliam:

What is less clear is what exactly constitutes effective classroom assessment. Although the studies cited above indicate that assessment for learning can improve learning, several studies have found conflicting results. For example, in a study of 32 fifth-grade teachers in Germany, Helmke and Schrader (1987) found that teachers who had an accurate knowledge of their students (as measured by the teachers’ ability to predict achievement test scores) were associated with higher levels of achievement only when the teachers also showed a high range of instructional techniques. Students taught by teachers who had a high knowledge of their students’ achievement but lacked a range of instructional techniques actually performed worse than students taught by teachers who did not know their students’ achievement. This study seems to indicate that collecting data if one cannot do anything with it is counterproductive.

Furthermore, even when teachers do manage to use information about student achievement to adjust or individualize their instruction, teachers may lack the ability to do so effectively. For example, in a 20-week study of 33 teachers in elementary and middle schools, Fuchs, Fuchs, Hamlett and Stecker (1991) found that teachers who received feedback on the achievement of students with learning difficulties in their classes made more adjustments to their teaching programs than teachers not given this information. However, the achievement of these students was improved only when this feedback was accompanied by advice from a computerized “expert system”, because the teachers not given the feedback from the expert system tended to re-explain how to do problems with the same algorithms that had led to previous failure.

Source: Wiliam, Dylan. “Keeping Learning on Track: Classroom Assessment and the Regulation of Learning.” In Second Handbook of Mathematics Teaching and Learning. Edited by Frank K Lester. Greenwich, CT: Information Age Publishing, 2007. pp. 10-11. [PDF preprint]

Last January, I moved from Massachusetts to North Carolina, and started a new job as a Physics professor. Spring was largely transition, teaching one light course here and making several long trips back north to keep the research project there going. Then came the summer, with a greater-than-usual blitz of travel and urgent work.

This fall, I started here for real. Now I’m really teaching! (And quite a lot of work it is, too.)

I’ve taught before, sort of: lots of labs and discussion sections as a TA, an outdoor leadership program for high school students, short and long teacher professional development programs… but that’s not the same has having responsibility for a full-scale university “lecture” course with ~60 students and 3 contact hours per week.

This fall, I’ve been teaching Conceptual Physics, a general education course with 55-ish students, drawing from all four class years (most heavily from freshmen and seniors) and almost every major on campus except Physics. I have, of course, been using a classroom response system (CRS, a.k.a. “clickers”). I cannot imagine teaching a course even a third this size without it; it would be like teaching without a whiteboard or a data projector. It would like becoming deaf in the classroom.

It has been interesting to see how all the pedagogical theory that I and my colleagues have been developing has fared. It isn’t easy! I’m encountering many of the same difficulties that the high school teachers in our project have voiced — problems I’ve tried to help them resolve with all kinds of sage advice.

I am finding, of course, that it isn’t quite as easy as I’d thought. (I’m envisioning many of our teachers nodding with a small smile of vindication, and thinking “See?” Touché.) It’s not so much that I’ve been wrong, as that I’m seeing new dimensions and nuances to the problems and the solutions. In fact, having to go through many of the same CRS learning issues as my teachers is really quite instructional.

I’ll try to document some of those insights in forthcoming blog posts. One question that I’m wrestling with, however, is this: just how much should I “let it all hang out” in a public forum that my students might quite possibly discover?

Maybe I shouldn’t worry about that. I find that my faith in my basic pedagogical principles and outlook are being strengthened, not weakened, by the experience. If anything, I’m entertaining even more radical thoughts about how we can re-envision the educational enterprise. (That might scare some of you who know me well.)

Anyway, the next post will focus on my latest insight about what makes creating good CRS questions difficult, at least for me. Hint: It’s not actually about the questions, though it initially appears that way.

A thoughtful post by sciencegeekgirl.

My team — the UMass Physics Education Research Group (UMPERG) — is moving! The University of North Carolina at Greensboro wants to build a robust, interdisciplinary, inter-departmental effort in science and math education research, outreach, and teacher preparation, and they’re serious enough to put some resources behind it.

To kick things off, they’ve hired my mentor and my group’s senior member, Bill Gerace, to fill the newly-created post of Helena Gabriel Houston Distinguished Professor of Science Education (a chair generously endowed by the C. D. Spangler Foundation — thanks, folks). He retired from UMass and began at UNCG this past summer. I’m moving in January, and my other colleague, Bill Leonard, starts at UNCG next summer. Whee!

(For those of you not fully up-to-date on UMPERG’s status, Jose Mestre moved to UIUC a few years ago, and Bob Dufresne has been primarily occupied with his Reading Recovery publishing business for a while.)

We’ve got the ambition and the mandate to do some big things at UNCG, such as building an inter-departmental “center” for science and mathematics education research and outreach, and creating a graduate degree program in Physics Education Research and/or Science Education Research. So, if you’re a potential graduate student, sabbatical visitor, collaborator, or faculty hire — or if you’re just curious — stay tuned.

The inevitable disruption is a partial explanation for my relative silence on this blog of late. 2008 has been a very, very tumultuous year, in both good and bad ways, for many reasons. My list of ideas for blog topics is getting longer, though…

Ask an educational researcher for his “teaching philosophy” and you’re likely to get a puzzled look and a long pause. These can be interpreted as “How do I condense years of research, literature reading, and theoretical development into a short answer?”

grounding

My philosophy of teaching draws from several research and philosophical traditions, as well as from the teaching experiences of myself and my colleagues. First and foremost, I am a constructivist (von Glasersfeld, 2007; Peschl, 2006). That term means many things to many people, but to me it means simply that knowledge and understanding cannot be “transmitted” between people; it must be constructed over time by each individual. In other words, learning is a deliberate process of sense-making that inevitably includes times of confusion, struggle, and reconciliation of difficulties. This relatively simple recognition has deep implications for instruction.

One implication is that communication cannot be taken for granted. All communication involves the sending of symbols that have no inherent meaning; meaning is intended by the sender and inferred by the recipient, and what meaning the recipient infers depends on his or her pre-existing expectations, assumptions, model of the sender, knowledge, and so on. As a teacher, that means I cannot presume that my spectacularly clear explanations communicate to the students what I intend them to. I need to model their interpretations as they model my intentions, and I need to “close the loop”, asking them to communicate back to me what they think they understood.

Another implication is that I do not “teach” so much as engineer a productive environment and set of stimuli for students to learn within, and provide coaching as they do so. (Note that this does not mean that lecture, or direct explanation, is always bad. Sometimes it is the appropriate stimuli to provide; nevertheless, I must remember that such lectures or direct explanations are not simply absorbed, understood, and immediately ready for future use.) Vygotsky’s notion of the zone of proximal development (Vygotsky, 1978) — that productive learning occurs within the space of challenges that students can succeed at with scaffolding, but not alone — suggests that I must continually tune the learning environment to students’ evolving capacities.

A third implication of constructivism is that students do not enter my classroom as blank slates; pre-existing knowledge, perceptions, perspectives, and experiences shape the understandings they construct in response to the environment and stimuli I provide. Thus, attempting to model their initial state, and track its subsequent evolution, is as important a component of my teaching job as designing my instruction.

Out of this perspective has grown the conceptual change tradition of educational research (Scott, Asoki & Leach, 2007), which studies the mechanisms by which students’ understanding of concepts evolves, the role of “misconceptions” in learning, and the like. More recent research, in what might be termed the knowledge in pieces tradition (Scherr, 2007), suggests that attending to what knowledge elements students have access to, and what contextual elements help to activate them, is more productive than considering what knowledge (or misconceptions) they “possess” (Redish, 2003; Hammer, Elby, Scherr & Redish, 2004; Dufresne, Thaden-Koch, Gerace & Leonard, 2005).

I am not just a constructivist, but a social constructivist informed by the sociocultural tradition of educational research (Carlsen, 2007; Mortimer & Scott, 2003). I see social interaction as essential to the internal knowledge-construction process, including student-to-student interactions as well as instructor-to-student ones. As Vygotsky observed (1987), the tools students use for internal cogitation are appropriated from social interactions. This implies that the classroom should be a place for exhibiting and exploring modes of thinking and argument, where students can see the process of “thinking science” modeled and where they can try it out themselves. Also, as Bakhtin observed (summarized in Wertsch, 1991, pp. 93-118), learning science largely means learning the social language of science (including conventions for thought and argument as well as vocabulary and grammar), and students must practice speaking a language to develop fluency. Thus, the classroom should be a place for students to practice “talking science”, with enough scaffolding from me to help them along, but not so much that I do the talking instead of them. In the very act of struggling to articulate their fuzzy thinking, students clarify their understanding of what they know, identify what they don’t, and often reach insights.

This has strong implications for what should occur in my classrooms. I do not see the classroom as a place for the dissemination of declarative content knowledge or the exhibition of proofs; those are more efficiently done through textbooks, multimedia, or other online resources. My classroom should be a place for dialogue and interaction, for exploration and confrontation and resolution. (In a large lecture hall, this is greatly facilitated by use of a classroom response system.)

My outlook is also shaped by the literature on student motivation and self-regulation (Koballa & Glynn, 2007; Wilson, 2006), and on the significance of students’ epistemological framing of the learning activities they engage in (Hammer, 1996; Hammer & Elby, 2003). Students are not black boxes, to whom instructional stimuli are applied and learning results; how they engage in learning activities matters tremendously, and as an instructor I must probe, model, monitor, and seek to influence that.

principles

Over time, I have distilled the practical implications for these (and other) pedagogical positions and educational research findings into four principles to guide instruction. These principles form the core of the “technology-enhanced formative assessment” (TEFA) pedagogy that my colleagues and I promote through in-service teacher professional development, and I would adhere to them in my own teaching.

The first principle is “Motivate and focus student learning with question-driven instruction.” This means posing tough, rich, meaty, often messy questions to students in order to contextualize and motivate subsequent learning, and often in order to catalyze or precipitate learning. It is grounded in the conceptual change tradition. It is motivated by an understanding that students perceive, process, and store information differently in response to a need, and that they “get” ideas by wrestling with the application of those ideas (Bransford et al., 1999, p. 139).

The second principle is “Develop students’ understanding and scientific fluency with dialogical discourse.” This means engaging students in discussion in which multiple ideas and ways of thinking are explored and contrasted, and in which students articulate and explore their own thinking. It is grounded in the sociocultural tradition.

The third principle is “Optimize teaching and students’ learning with formative assessment.” This means making students’ knowledge and thinking visible in order to adjust and optimize subsequent learning and teaching. It is motivated by an understanding that effective instruction requires detailed and current information about the specific students being taught, and that effective learning requires accurate self-knowledge (Wiliam, 2007). According to a seminal literature review by Paul Black and Dylan Wiliam (1998), “innovations” involving formative assessment produce learning gains that are among the largest ever found for educational interventions.

The fourth principle is “Help students cooperate in the learning process and develop metacognitive skills with meta-level communication.” This means communicating about communication, about cognition, about learning, and about the purposes of instructional experiences. It is grounded in literature on student motivation and self-regulation. It is motivated by an understanding that learning works better when students frame their participation appropriately and understand what they are supposed to be paying attention to.

I do not consider these four principles to be independent and arbitrary beliefs; they interlock and reinforce each other in a highly synergistic way. This can be seen in the way they are enacted in the TEFA “question cycle” — one specific way out of many of realizing the principles — which has been described elsewhere (Dufresne et al., 1996; Beatty, Leonard, Gerace & Dufresne, 2006).

references

Beatty, I. D., Leonard, W. J., Gerace, W. J., and Dufresne, R. J. (2006). Question driven instruction: Teaching science (well) with an audience response system. In Banks, D. A., editor, Audience Response Systems in Higher Education: Applications and Cases. Idea Group Inc., Hershey, PA.

Black, P. and Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1):7-74.

Bransford, J. D., Brown, A. L., and Cocking, R. R. (1999). How People Learn: Brain, Mind, Experience, and School. National Academy Press, Washington, D.C.

Carlsen, W. S. (2007). Language and science learning. In Abell, S. K. and Lederman, N. G., editors, Handbook of Research on Science Education, chapter 3, pages 57-74. Lawrence Erlbaum Associates, Mahwah, NJ.

Dufresne, R. J., Gerace, W. J., Leonard, W. J., Mestre, J. P., and Wenk, L. (1996). Classtalk: A classroom communication system for active learning. Journal of Computing in Higher Education, 7:3-47.

Dufresne, R. J., Thaden-Koch, T., Gerace, W. J., and Leonard, W. J. (2005). Knowledge representation and coordination in the transfer process. In Mestre, J. P., editor, Transfer of Learning from a Modern Multidisciplinary Perspective, chapter 5, pages 89-119. Information Age Publishing.

Hammer, D. (1996). More than misconceptions: Multiple perspectives on student knowledge and reasoning, and an appropriate role for education research. American Journal of Physics, 64:1316-1325.

Hammer, D. and Elby, A. (2003). Tapping epistemological resources for learning physics. Journal of Learning Sciences, 12:53-90.

Hammer, D., Elby, A., Scherr, R. E., and Redish, E. F. (2004). Resources, framing, and transfer. In Mestre, J. P., editor, Transfer of Learning: Research and Perspective. Information Age Publishing, Greenwich, CT.

Koballa, T. R. and Glynn, S. M. (2007). Attitudinal and motivational constructs in science learning. In Abell, S. K. and Lederman, N. G., editors, Handbook of Research on Science Education, chapter 4, pages 75-102. Lawrence Erlbaum Associates, Mahwah, NJ.

Mortimer, E. F. and Scott, P. H. (2003). Meaning Making in Secondary Science Classrooms. Open University Press.

Peschl, M. F. (2006). Modes of knowing and modes of coming to know: Knowledge creation and co-construction as socio-epistemological engineering in educational processes. Constructivist Foundations, 1(3):111-123.

Redish, E. F. (2003). A theoretical framework for physics education research: Modeling student thinking. In Vicentinni, M. and Redish, E. F., editors, Proceedings of the Varenna Summer School, “Enrico Fermi” Course CLVI. IOS Press, Amsterdam.

Scherr, R. E. (2007). Modeling student thinking: An example from special relativity. American Journal of Physics, 75(3):272-280.

Scott, P., Asoki, H., and Leach, J. (2007). Student conceptions and conceptual learning in science. In Abell, S. K. and Lederman, N. G., editors, Handbook of Research on Science Education, chapter 2, pages 31-56. Lawrence Erlbaum Associates, Mahwah, NJ.

von Glasersfeld, E. (2007). Key Works in Radical Constructivism. Sense Publisherss.

Vygotsky, L. S. (1978). The development of higher psychological processes. Harvard University Press.

Vygotsky, L. S. (1987). Thinking and speech. In Rieber, R. W. and Carton, A. S., editors, The Collected Works of L. S. Vygotsky. Plenum Press.

Wertsch, J. V. (1991). Voices of the Mind: A Sociocultural Approach to Mediated Action. Harvard University Press.

Wiliam, D. (2007). Keeping learning on track: Classroom assessment and the regulation of learning. In Lester, F. K., editor, Second Handbook of Mathematics Teaching and Learning, pages 1051-1098. Information Age Publishing, Greenwich, CT.

Wilson, T. D. (2006). The power of social psychological interventions. Science, 313:1251-1252.

Suggestions and opinions are welcome!

I probably need help with my Facebook profile, too.

I’m an educational researcher by profession. I tend to believe that I know a lot about how to teach well, especially physics. I’ve read the literature, attended the conferences, conducted research, engaged in countless discussions about teaching and learning, and published some papers. Yes, I’ve even designed and taught physics courses, though not much since finishing my Ph.D. (I’m on a research position, not a teaching one. Unfortunately.)

Which is why the following anecdote is acutely embarrassing.

This past July, my colleague, group leader, and travel buddy Bill Gerace and I spent two unexpectedly hot, humid weeks in Vitznau, Switzerland. We went to teach physics to hospitality management students (as we did in Singapore the previous summer). Does that seem bizarre? UMass has partnered with hospitality management schools in Singapore and Vitznau to offer a UMass baccalaureate. Students must fulfill regular UMass degree requirements, including “distribution” criteria of so many literature classes, so many science classes, etc. The partner schools used to ship their students over to UMass for a year or so to take all those courses, but someone figured out that it’s cheaper to send UMass faculty over to teach two-week intensive courses in various subjects. So, UMass asks its faculty for volunteers.

Knowing a good thing when he sees it, Bill jumped on the opportunity. He used the stipend to pay my travel expenses, so we both went more or less for free, inveterate travel junkies that we are. Bill taught, I helped out with computer tasks and improvised experiment/demo equipment, and I telecommuted to fulfill the duties of my “real” job. (Lest you think I’m a slacker, know that we committed to this trip before the big research grant providing my real job had been awarded.)

Back to the humility thing. A few days into the course, we reached the topic of “conservation of energy.” I have a way of explaining the concept that I think makes a whole lot of intuitive sense and should be brilliantly clear to students, so I asked Bill if I could teach that segment. He agreed, and I did. I tried to, anyway.

So I started, and introduced my analogy between conservation of energy and financial accounting, making the point that money is never created or destroyed, but moved from one account to another, to cash in your pocket, to credit (or less debt) on your credit account, etc. This is is just like energy: it gets shifted around from one form to another, one “place” to another, but the total amount remains the same. (Nobody asked about governments that print money.) This should be really accessible to students also taking management classes, right?

As it goes on, I get increasingly uncomfortable. Eyes are glazing over. A crunch on classroom space has pushed us into the computer lab for this class, and more than a little key-pecking and monitor-glancing is happening. I ask questions and get very little response; the answers I do get are tentative and unsure, more like guesses than opinions.

And then it hits me. I’m doing it: the classic IRE triadic pattern of classroom discourse, in which the instructor “initiates” with a question, the students “respond” with an answer, and the instructor “evaluates” the correctness of the response. No “uptake” or chaining of responses to responses, no true dialogic discourse or exploration of points of view. This is quizzing, not discussion. I’ve just read an entire damn book about patterns of discourse, nodding in agreement as the authors expounded upon the futility of IRE-based teaching, and here I am torturing perfectly nice foreigners with it.

It’s not that I don’t really understand the theory or the arguments against IRE. I very much do, to the point that it seems self-evident. Rather, IRE-style teaching is so deeply ingrained in me from 20-odd years of being a student (not counting preschool or the interminable stretch of my dissertation work) that I fell into it without even thinking.

So I bailed. I tag-teamed off to Bill almost mid-sentence. No one can improv physics like Bill, so he picked up smoothly and continued the lesson (with significantly less IRE and eye-glazing).

Licking my wounds later and reflecting on the experience, I realized I had been doomed from the very moment I first desired to teach that lesson. I began by thinking about what was inside my head — the cool analogy I was going to make — rather than about what was inside the students’ heads. Rule #1 of teaching:

It doesn’t matter what comes out of your mouth (or shows up on your PowerPoint slides). All that matters is what happens in the students’ minds, so find out what that is and interact with it.

The myth of coverage is a corollary of this.

The morals of this story?

1. There’s a huge gap between knowing and doing. We generally do what we’re patterned on, not what we would choose if we thought about it. Especially under stress or on the spot.
2. If we really want to impact the way science (or anything else) is taught, we must change the formative learning experiences of our future teachers. It’s a bootstrapping problem.
3. Don’t lose sight of the goal for even a moment: in this case, developing students’ understanding. Teaching cleverly is not synonymous with making learning happen.
4. Self-monitoring and reflection are very powerful learning tools. I learned more from that one experience than from dozens of learned papers and discussions. (Bill likes to say that “All learning is through trauma.” He’s using learning in a narrow, strong sense and trauma in a general, cognitive one.)

(If you click on the title links, the relevant “slide” should open in a new browser window. Subsequent slides should open in the same window, so if you resize the window to something about 1024 x 768 and drag it to the side of the narrative in this window, you should somewhat recreate the effect of the talk. Except for the part about me being rattled by technical problems and talking way too fast. Hopefully, your display is working better than my projector did during the talk. The slides are HTML, not PDF or images, so fonts and layout and such may vary.)

Title & Authors

Good evening.

This is the kind of talk where I tell you about a current research project, because it’s good for professionals to know what their colleagues are up to. It’s also the kind of talk where I slide in my own pedagogical opinions, because, well, I want to change the world.

If you hear anything brilliant in my talk, credit probably belongs to my colleagues in the UMass Physics Education Research Group. They’ve been thinking about this for longer than I.

Classroom Response Systems

How many of you know what a “classroom response system” is? Also known as an “audience response system”, “voting machine”, “polling system”, or “clicker system”? [Probe audience and adjust talk as appropriate.]

Briefly, for those of you who haven’t: A classroom response system provides a supplemental, technology-mediated channel of communication between instructor and students. [1]

It is a combination of hardware and software that allows an instructor to:

* Present a question to students in class;
* Have them submit answers;
* Immediately aggregate the responses; and
* Share the results with the whole class, usually as a histogram.

[Pause.] Are classroom response systems effective?

[Pause.] Please raise your hand if you think they’re educationaly effective… Who thinks they’re not so hot?… Who’s still sitting on the fence?…

That was a trick question.

Assessing Instructional Technology

This session is about “assessing the educational effectiveness of technology”, but technology doesn’t have educational effectiveness. At least not by itself.

What is the culinary effectiveness of a wok?

[Pause.] If you know how to use one and you’re trying to make a nice stir-fry, it’s quite effective. But if you have no clue what you’re doing in a kitchen, or you’re trying to make a quiche, it’s pretty darn useless.

Any evaluation of instructional technology must ask four questions:

* For what purpose is the technology being applied?
* How is it being applied?
* How well is the user doing it?
* How well does the technology enable or enhance the attempt?

Only the fourth is about the technology itself.

[Pause] That perspective motivates the project I’ll be describing.

Uses for Classroom Response Systems

Classroom response system can be tremendously powerful instructional tools, but they’re not a silver bullet. We’ve seen them used completely ineffectually, horribly abused, or used well for ends we don’t see much merit in.

Some of the different goals people use classroom response systems for include:

* Taking attendance
* Administering quizzes
* Provoking engagement [2]
* Checking for progress
* Promoting knowledge diffusion [3]

All of these typically involve sprinkling response system questions throughout “normal” instruction. The approach we at UMPERG have developed is, I think, considerably more radical. [4]

The Question Cycle

The core idea is that we structure class time around an interactive question cycle [5], iterated three times per hour, more or less. The question cycle serves as the primary vehicle, the primary engine, for instruction. It’s not an add-in; it’s the meat of the class.

We use the question cycle to:

* Reveal and explore students’ thinking,
* Introduce new ideas,
* Refine and extend students’ understanding, and
* Develop students’ analysis and problem solving skills.

A classroom response system is not strictly necessary for this, but it sure helps.

Classroom Dynamic

There’s a lot more to our approach than the question cycle, but that would require a few more ten-minute talks.

What do we call it?

We’ve called the approach by various names at various times, depending on what aspect we’re focusing on.

* Question Driven Instruction (QDI)
* Technology Enhanced Formative Assessment (TEFA)
* Assessing to Learn (A2L)
* Agile Teaching (AT)

I’ll use the first one, “Question Driven Instruction” or “QDI”, tonght.

So back to the project. We want to make a really strong case for the efficacy of classroom response system coupled with the QDI pedagogical approach. That means doing a large-population study of student learning impacts, a so-called “scaling study”, rigorous enough for the What Works Clearinghouse.

But, we can’t evaluate learning impacts fairly without a cadre of teachers who are trying to do QDI, and doing it competently.

Scaling Study Preliminaries

So, we need two things first.

* We need a professional development program that can efficiently and reliably move teachers to a state of QDI competence.
* And we need measures of implementation fidelity that tell us when a teacher is, in fact, doing a reasonable job at it.

If we don’t have the first, we won’t have any QDI to study. If we don’t have the second, we won’t know whether negative results — that is, poor learning impacts — indicate that QDI doesn’t work, or that it just isn’t happening.

So the project we proposed to The National Science Foundation, and that we’re currently working on, is a preliminary study that sets up a scaling study.

TLT Project Goals

Our project is called Teacher Learning of Technology Enhanced Formative Assessment. We are studying teacher learning, not student learning. There will be no measuring of student learning gains.

The project has three general goals:

# To better understand how teachers get from novice to expert in the use of a classroom response system and QDI.
# To refine our methodology for teaching QDI to teachers, and “package” it so that others can do the professional development.
# To prepare the measures, instrumentation, design, and general know-how for a “scaling study” on student learning impacts.

TLT Design: Professional Development

Our plan is to work with the entire science department at a high school, so that teachers can learn collaboratively and support each other, and so that students get a consistent learning experience from class to class.

During the first treatment year, we’ll conduct an intensive two-semester professional development course. The course will focus on the skills that go into successful QDI, including:

* Using classroom response system technology,
* Designing effective questions,
* Navigating the question cycle,
* Moderating classroom discourse, and
* Integrating QDI with curriculum goals and external constraints.

[Pause.] Change doesn’t happen overnight. For all three treatment years, we’ll facilitate a collaborative action research program for the teachers. This will:

* Support the ongoing evolution of their teaching practice, and
* Provide a forum where the teachers can set the agenda and come to terms with QDI.

TLT Design: Data Acquisition

We’ll collect longitudinal data on teacher change over the three treatment years, plus baseline measurements. We’ll gather data from classroom observations, and from interviews and surveys, in order to track changes in:

* What teachers do in the classroom;
* How they approach lesson planning;
* How they perceive knowledge, learning, and teaching;
* What aspects of teaching occupy their attention most;
* What difficulties they wrestle with;
* What supports and assistance they find helpful;
* How they perceive their own QDI skills; and
* How their students perceive the classroom environment.

We’re developing most of the research instruments ourselves, since existing instruments don’t really address the variables we want to track. We are, however, incorporating pieces of established instruments to aid comparison with other research.

TLT Timeline

We’ll be working with two schools, staggered by one year so that we’re taking baseline data in school 2 while teaching the PD course in school 1.

Our first school is fairly small, with eleven participating teachers. This includes most of the high school science faculty, as well as some of the math and junior high science teachers. We’re planning on a larger cohort for school 2, with over 20 participants.

Where are we now? We’re just wrapping up baseline data collection for school 1. Next week, we’ll kick off the PD course with a three-day workshop. We don’t have any results yet, but come back next year for some fascinating preliminary findings!

Links & Credits

If you want to learn more about the project, come chat with me and see our poster during tomorrow morning’s poster session.

Or, visit our web page about the project. (It’s a little sketchy right now, but I’ll be augmenting it soon.)

If you fell asleep in the middle of this talk and want to see what you missed, I’ll post the narrative and slides to my personal weblog, hopefully tomorrow.

And by the way, our graduate student Colin Fredericks is giving a talk at ten tonight, right here, about a different but also interesting project. So please stick around a little longer.

Thanks for your time. Any questions?

¹ Ian D. Beatty (2004) Transforming Student Learning with Classroom Communication Systems. Educause Center for Applied Research (ECAR) Research Bulletin ERB0403.

² Robert J. Dufresne, William J. Gerace, William J. Leonard, Jose P. Mestre, Laura Wenk (1996). Classtalk: A Classroom Communication System for Active Learning. Journal of Computing in Higher Education 7, 3-47.

³ Mazur, Eric (1997) Peer Instruction: A User’s Manual (Prentice Hall, Upper Saddle River, NJ).

⁴ Ian D. Beatty, William J. Leonard, William J. Gerace, Robert J. Dufresne (2006). Question Driven Instruction: Teaching science (well) with an audience response system. In David A. Banks (Ed.), Audience Response Systems in Higher Education: Applications and Cases (Idea Group Inc., Hershey, PA).

⁵ Robert J. Dufresne, William J. Gerace, Jose P. Mestre, William J. Leonard (2000). ASK-IT/A2L: Assessing Student Knowledge with Instructional Technology. UMass Scientific Reasoning Research Institute technical report Dufresne-2000ask.

So why was I surprised to find that Uganda is different?

The geographically erudite reader will note that the African countries I’ve previously visited are all in the southernmost portion of Africa, more or less between 17 and 34 degrees south latitude. Uganda is smack on the equator. My southern African countries all have a history of colonial rule, and the one I’m most familiar with — South Africa — is still recovering from the brutality of Apartheid. Uganda was a British protectorate, never a colony. Uganda has different ethnic groups, different languages.

Uganda is different. Duh!

If I stopped there, this essay would be a simple self-smacking of the forehead. In the “Think Twice” spirit, however, I’m going to dig a little deeper. Is there a moral here, aside from the always-apt “beware the trap of hubris”?

I think there is, and it has to do with the nature of knowledge, and the many kinds of knowing. I “knew” in an abstract, conceptual, and logical sense that Africa is variegated, but not in a deep enough way to affect my unexamined expectations. Perhaps this parallels the distinction between “passive” and “active” vocabulary. (A person’s “passive” vocabulary with a language is all the words she understands when she hears or reads them. Her “active” vocabulary is all those that come to mind, unprompted, for use when speaking or writing.)

Educational researchers know (heh) that there are many kinds and degrees of knowing, and that we don’t fully understand all that’s involved in the thing we blithely call “knowing” [Redish-2003tfp]. It’s complicated. For useful knowledge, we need to have the right “mental resources” in our heads, and we also need to have the right associations and triggers in place so that those resources are “activated” in the appropriate contexts.

When do I understand the concept of “force”? When I can spell it? When I can quote a definition? When I can recognize the presence of one in simple and familiar situations, or in subtle and novel situations? When I can use the concept as a tool to reason with in familiar contexts? In unfamiliar ones? I doubt there’s a person on the planet who can do all of these things in all possible cases, infallibly, so does anybody really understand “force”?

If that doesn’t hurt your brain enough, consider metacognitive knowledge: knowledge about your own knowledge. To quote my colleague and former dissertation advisor, Bill Gerace: “Sometimes you know something. Sometimes you know you know something. And sometimes you know you knew something, but don’t any more.” What’s going on there?

And, as Uganda has reminded me, sometimes you only think you know something.