• http://bit.ly/beatty-aapt-2010

Update: Here’s another link, in case bit.ly goes under:

• http://goo.gl/XPgM

Timing: towards the close of a unit on magnetism, after gravitation and electrostatics have been taught. (No guarantees that they’ve been learned, though.)

Question: If you were a superhero, which power would you rather have?

1. Change the mass of things.
2. Change the charge of things.
3. Change the magnetization of things.

Why is this a great question?

For one thing, it’s pretty obvious that even if the instructor might have a preferred answer (which he didn’t), there’s no “correct” answer in an absolute sense. So, students can relax a little and explore what they actually think.

I generally run this one without a small-group discussion phase before the poll. When the poll has been taken and students begin voicing their choices and reasons in the whole-class discussion phase, it rapidly becomes clear that there are many different ways to think about the question.

And then the shift happens. Students stop thinking about their goal as “come up with the most correct thing to say” (or worse, “come up with the thing the instructor wants to hear”), and start thinking about it as “come up with the most clever thing to say”. The interaction has been reframed. Score!

Somebody picks “mass” so that they could make a bullet harmless by dropping its mass to near-zero, or so they could throw a pebble and then increase its mass hugely so that it would punch through a wall. Someone else says “Wait a minute, if you increased its mass, would its velocity stay the same, or would its velocity become tiny so that its momentum was conserved?” Everyone looks at me, and I shrug and say “I guess it depends on how this superpower works, eh?” I couldn’t care less how it would work, but I’m very happy that the connection between mass, inertia, and momentum has been articulated. Score!

Someone else says they picked “charge” because they could make lightning zap things. I ask if anyone picked “charge” for a different reason, and someone else says that if they could control charge, they could make things attract or repel, which means they could make things (including themselves) levitate or fly. “Mass affects gravity, which is only for attracting; charge affects the electric force, which can attract or repel.” Score!

Then another student piggybacks on that, saying “But wait, the magnetic force can make things twist and turn as well as attract or repel. Wouldn’t that be more useful?” Someone replies “Huh?”, and a short clarification dialog ensues. I smile. Then a student asks whether the power means that non-magnetic things could be made magnetic, or only that magnetic things could be made more or less magnetic, or have their polarities switched, or what. I shrug again, happy that the distinction has been voiced. Score!

Then one student who’s been quiet all along speaks up. She says “I think I want to control charge, because that’s what brain cells use to communicate, so I could alter people’s thoughts. Maybe I could alter computer programs, too.” Eyes widen throughout the room. Score!

In the resulting silence, I innocently inquire whether she’d need to be able to sense all the charge patterns flowing around — and even harder, interpret them — to make that power useful. She looks nonplussed, and then says “Well, at least I could scramble someone’s head pretty well, maybe give them amnesia!” Laughter.

The actual discussion doesn’t flow quite this smoothly and efficiently, of course; a fair number of less interesting, or less defensible, or less comprehensible assertions are made, and I do a little prompting and steering to bring out some of these points. Nevertheless, I’ve accomplished three very important tasks: We’ve compared and contrasted gravity, the electrostatic force, and the magnetic force, and the roles that mass, charge, and magnetism play within those; I’ve engaged the students in creative, open-ended thinking to apply abstract physics ideas to real-world (okay, comic-book-world) things; and I’ve gotten the students to enjoy physics class. Triple score!

This one question nicely instantiates all four principles of Technology-Enhanced Formative Assessment (TEFA):

• Motivate and focus student learning with question-driven instruction (QDI);
• Develop students’ understanding and scientific fluency with dialogical discourse (DD);
• Inform and adjust teaching and learning decisions with formative assessment (FA); and
• Help students develop metacognitive skills and cooperate in the learning process with meta-level communication (MLC).

For more about TEFA (probably far, far more than you really want to know), see Beatty & Gerace (2009), Technology-Enhanced Formative Assessment: A research-based pedagogy for teaching science with classroom response systems, Journal of Science Education and Technology 18(2):146-162.

BTW, my inspiration for this question was a biology question by Cathy Wanat of Northampton (MA) High School (since retired). It showed a photograph of a long buffet table loaded with different food dishes, with lines of people moving along both sides as they added food to their plates. The question was “Which of the following is most like this picture?”, and the answer choices were various parts of the digestive system: mouth, esophagus, stomach, small intestine, etc. She said the resulting class discussion was mind-blowing. Thanks, Cathy!

An intriguing research result is that formative assessment may actually be counterproductive if the teacher doesn’t have adequate strategies for responding to that information. Here’s a quote about that from a paper by Dylan Wiliam:

What is less clear is what exactly constitutes effective classroom assessment. Although the studies cited above indicate that assessment for learning can improve learning, several studies have found conflicting results. For example, in a study of 32 fifth-grade teachers in Germany, Helmke and Schrader (1987) found that teachers who had an accurate knowledge of their students (as measured by the teachers’ ability to predict achievement test scores) were associated with higher levels of achievement only when the teachers also showed a high range of instructional techniques. Students taught by teachers who had a high knowledge of their students’ achievement but lacked a range of instructional techniques actually performed worse than students taught by teachers who did not know their students’ achievement. This study seems to indicate that collecting data if one cannot do anything with it is counterproductive.

Furthermore, even when teachers do manage to use information about student achievement to adjust or individualize their instruction, teachers may lack the ability to do so effectively. For example, in a 20-week study of 33 teachers in elementary and middle schools, Fuchs, Fuchs, Hamlett and Stecker (1991) found that teachers who received feedback on the achievement of students with learning difficulties in their classes made more adjustments to their teaching programs than teachers not given this information. However, the achievement of these students was improved only when this feedback was accompanied by advice from a computerized “expert system”, because the teachers not given the feedback from the expert system tended to re-explain how to do problems with the same algorithms that had led to previous failure.

Source: Wiliam, Dylan. “Keeping Learning on Track: Classroom Assessment and the Regulation of Learning.” In Second Handbook of Mathematics Teaching and Learning. Edited by Frank K Lester. Greenwich, CT: Information Age Publishing, 2007. pp. 10-11. [PDF preprint]

Of the forty-some teachers we’ve worked with to some degree or another, by far the number one difficulty they’ve reported is the challenge of regularly creating effective clicker questions to use in class. The characteristics that make a question “work” — meaning engage students in quality classroom discussion and promote learning — are not obvious, and typical back-of-the-chapter or quiz-type questions will fail miserably. In the project’s professional development meetings, we’ve spent a great deal of time talking about question creation, and I’ve developed various frameworks in an attempt to help make it more science and less art.

This semester, in prepping my own Conceptual Physics class, I’ve run into exactly the same difficulty. “Today I’m teaching topic X, and I need some good questions. Um, ah, hmm…” Not so easy, even with all the frameworks and such.

One flash of insight I had recently is that, at least for me, it’s not really creating questions that’s tough. The hard part is figuring out what I want my students to learn from the class, and casting that in terms of what I want my students to be able to do. I’ve been trying to shift my thinking from “the material” to “the demonstrable, assessable learning outcomes” (cf. The Myth of Coverage).

Once I can articulate what I would like my students to be able to do after the class, it’s generally relatively easy to invent a few good clicker questions. I just formulate a question asking them to do that (in a particular context), and then much of the class activity is me helping them struggle through the process as they learn how. (This is the principle we’ve called “Question-Driven Instruction”, as articulated in Beatty & Gerace 2009 and elsewhere.)

Which all means that when someone says “Creating good clicker questions is hard”, I’m now inclined to hear that as “Thinking in terms of demonstrable student learning outcomes rather than topic coverage is hard.” And I agree. I also think it’s one of the many desperately needed shifts to how we conceive of this whole enterprise we call organized schooling.

I’m not saying that this is the only difficult aspect of creating good questions, but it’s definitely key for me. I’m curious what others think. If you’ve taught with a classroom response system, what do you think? Does that ring true? Do you have any similar or conflicting experiences to share? Comments are open…

Last January, I moved from Massachusetts to North Carolina, and started a new job as a Physics professor. Spring was largely transition, teaching one light course here and making several long trips back north to keep the research project there going. Then came the summer, with a greater-than-usual blitz of travel and urgent work.

This fall, I started here for real. Now I’m really teaching! (And quite a lot of work it is, too.)

I’ve taught before, sort of: lots of labs and discussion sections as a TA, an outdoor leadership program for high school students, short and long teacher professional development programs… but that’s not the same has having responsibility for a full-scale university “lecture” course with ~60 students and 3 contact hours per week.

This fall, I’ve been teaching Conceptual Physics, a general education course with 55-ish students, drawing from all four class years (most heavily from freshmen and seniors) and almost every major on campus except Physics. I have, of course, been using a classroom response system (CRS, a.k.a. “clickers”). I cannot imagine teaching a course even a third this size without it; it would be like teaching without a whiteboard or a data projector. It would like becoming deaf in the classroom.

It has been interesting to see how all the pedagogical theory that I and my colleagues have been developing has fared. It isn’t easy! I’m encountering many of the same difficulties that the high school teachers in our project have voiced — problems I’ve tried to help them resolve with all kinds of sage advice.

I am finding, of course, that it isn’t quite as easy as I’d thought. (I’m envisioning many of our teachers nodding with a small smile of vindication, and thinking “See?” Touché.) It’s not so much that I’ve been wrong, as that I’m seeing new dimensions and nuances to the problems and the solutions. In fact, having to go through many of the same CRS learning issues as my teachers is really quite instructional.

I’ll try to document some of those insights in forthcoming blog posts. One question that I’m wrestling with, however, is this: just how much should I “let it all hang out” in a public forum that my students might quite possibly discover?

Maybe I shouldn’t worry about that. I find that my faith in my basic pedagogical principles and outlook are being strengthened, not weakened, by the experience. If anything, I’m entertaining even more radical thoughts about how we can re-envision the educational enterprise. (That might scare some of you who know me well.)

Anyway, the next post will focus on my latest insight about what makes creating good CRS questions difficult, at least for me. Hint: It’s not actually about the questions, though it initially appears that way.