(I still think I’ve got too many fine-grained and overlapping standards, though. That does cause headaches.)

One thing I’ve realized while developing assessments for this first “unit” of the course — kinematics and vectors — is that my choice of standards can make assessment harder or easier. Some specific realizations:

• Too many standards makes it hard for me to adequately assess, and re-assess, them all. Fewer is better (though too few loses the laser-sharp-feedback quality of SBG).

• Standards that are too “basic”, and which are necessary steps towards the harder ones, aren’t necessary to articulate as standards on their own; students must learn to do them anyway in order to do the higher-level ones. Having them in the list simply clogs up assessments. Example: I can draw or interpret motion diagrams (strobe diagrams).

• Standards that don’t “factor” cleanly are difficult to assess and give separate mastery ratings for. Example: I can use 2D/3D constant-acceleration kinematics (graphical analysis and/or formulae) to analyze an object’s motion, working with numbers or variables and it’s too-close cousin, I can use the projectile motion model to analyze physical situations. I can articulate a distinction between these two — make the second be about recognizing the independence of the two coordinates and the acceleration in each, and the first be about “doing” the subsequent kinematics — but it’s awkward and unclear.

• Some really seem to belong as part of a larger standard, not hanging out on their own. Example: I can determine or reason about an object’s instantaneous acceleration.

So now I’m rewriting my unit 2 standards. (With fewer standards in later units, I’ll have to add some weighting factors to avoid overly-counting kinematics in the final grade.)

Why? For one thing, I’m already a full day behind my planned schedule. Ugh. 50 minutes is so short! Something’s gotta give. I’m feeling the voices of 100+ standards screaming at me to assess, reassess, reassess!

For another, I think I’ve been so concerned about my SBG implementation that I’ve lost the forest for the trees. I forgot to introduce students to what physics is all about and why it’s worth studying, beyond a 3-minute definition (okay, three definitions, including “Whatever physicists feel like studying”). Wait, what’s the point of all these kinematical vector thingies?

You’d never know from my teaching that I recently read Perkins’ Making Learning Whole.

Pondering what to do on Wednesday (and thereafter) to save this course without looking incompetent and losing students’ confidence…

Some initial thoughts about my experiences with and realizations about SBG:

Choice of specific standards is absolutely critical, and one key choice is “grain size”. I could identify a few larger, more general capacities to assess (extreme example: “I can use work and energy ideas to analyze situations and solve problems”). Alternatively, I could unpack those into a plethora of highly targeted standards (“I can draw velocity vs. time graphs for constant-acceleration problems based on a motion diagram”, “I can draw acceleration vs. time graphs for constant-acceleration problems based on a motion diagram”, “I can draw acceleration vs. time graphs for constant-acceleration problems based on a velocity vs. time graph”, etc. etc. etc.). Somewhere in between these extremes is a sweet spot that optimally balances specificity of feedback to the student with practicality of assessment and tracking.

I seem to be on track to have a bit over a hundred standards in this course, at a rate of about 6-8 per chapter. That’s 3-4 per class meeting, more or less. That seems like a lot, and more than many other SBG practitioners seem to have — but I’m having a great deal of difficulty combining them into more coarsely-grained standards without doing violence to my sense of what the “things” to be learned really are. To put it another way: The topics seem to naturally cleave along certain lines, and allowing that gets me to where I am.

Despite that last sentence, standards can be divided along various lines, and different ways of grouping sub-elements can align more or less well with the organization of my textbook and accompanying workbook, easier ways of assessing, etc. I initially brainstormed a list of standards, but have been doing some refactoring as I went through and correlated them with textbook sections and daily class plans.

SBG drives me to assess (and reassess) EVERYTHING I want students to seriously try to learn, rather than allowing me to sample a subset of the learning goals. I suppose I could simply not assess some of the standards and let them drop out of the grading scheme, but I currently feel that if it’s on the standards list, I ought to assess it. And that’s a lot! Which leads to my next realization:

Articulating learning standards makes me much more aware of what I’m actually asking students to learn (more than I would be with a traditional by-topics list), and there’s a freaking lot of stuff for intro physics students to learn. Wow. No wonder physics is hard!

If I want a relatively simple grade calculation — each student gets a 0-4 mastery rating on each standard, and the final grade calculations consists of averaging all those ratings and then mapping to a letter grade — then the number of standards per general topic had better be proportional to the topic’s importance, since that determines its weight in the overall grade. I find it tempting to split early chapters into many fine-grained standards (e.g., specific kinematics graphing skills, specific types of motion, etc.), but leave later chapters as more holistic standards (use the Impulse-Momentum principle to analyze collisions). Unfortunately, that overly weights the early stuff. I can either weight different standards differently, or unpack the later standards into finer-grained components… which is probably beneficial to both me and the students, but darn, it’s hard work!

Unless I want to box myself into having to assess each standard multiple times, in different ways (for different levels of mastery), or having different mastery scales for different standards, I’d better construct my standards such that only one assessment probe is necessary for each. That can mean peeling “advanced” mastery levels off of the top end of the mastery rubric and creating new standards specifically targeting those. For example: Instead of having the top mastery rating be reserved for “Can recognize need to apply this within a complex scenario and figure out how to connect to other principles” (which takes a different exam question than “Can apply to a straightforward situation when prompted”), I can have a separate standard for “Identify which principle(s) apply to a complex situation” and “Combine multiple principles to solve a problem”. Put another way: If every standard has an “above and beyond” level, I need to assess every student for that level of mastery on every standard, and that’s probably unrealistic. Better to have a few explicit “above and beyond” standards.

Reassessment is the heart of SBG — it’s what makes assessment formative, and lets students learn from their mistakes and keep making progress — but it’s also looking like the hardest part to implement, at least in my context (60 students, three 50-minute classes per week, the fact that giving up my free afternoons/days to a stream of reassessing students would kill my research efforts). I’ve been very cagey about not promising anything specific about reassessment yet in this course, but I can’t keep that up much longer.

The other big question, of course, is whether students really will do the work –reading, workbook, homework, etc. — without having those be graded. Most students do end up in the trap of running from deadline to deadline, only focusing on whatever is “due” next and prioritizing tasks by grade impact.

Stay tuned. This is very much a work in progress.

I find myself wondering about the feasibility of some kind of two-tier system, where (1) authentic, multifaceted, ill-structured PBL-type performance tasks are unpacked into (2) component/requisite “learning standards”; the learning standards are individually assessed, re-assessed, and hopefully mastered; and the overarching PBL-type performance task is then completed and assessed in its own way. Somehow, both levels would contribute to feedback and grading.

But, I worry about ending up with some Frakensteinian horror when the two are grafted together. “80% of the credit for ultimate standard mastery, 20% for one-time project grades” seems antithetical to SBG, and inconsistent. Building an additional “level of mastery” onto each granular standard to indicate “successfully used in a project” seems kludgy, and poorly aligned with the holistic nature of PBL.

Thoughts from experienced SBG implementers — or anyone else, for that matter? (Preferably not about SEO, though. Thanks. )

Of the forty-some teachers we’ve worked with to some degree or another, by far the number one difficulty they’ve reported is the challenge of regularly creating effective clicker questions to use in class. The characteristics that make a question “work” — meaning engage students in quality classroom discussion and promote learning — are not obvious, and typical back-of-the-chapter or quiz-type questions will fail miserably. In the project’s professional development meetings, we’ve spent a great deal of time talking about question creation, and I’ve developed various frameworks in an attempt to help make it more science and less art.

This semester, in prepping my own Conceptual Physics class, I’ve run into exactly the same difficulty. “Today I’m teaching topic X, and I need some good questions. Um, ah, hmm…” Not so easy, even with all the frameworks and such.

One flash of insight I had recently is that, at least for me, it’s not really creating questions that’s tough. The hard part is figuring out what I want my students to learn from the class, and casting that in terms of what I want my students to be able to do. I’ve been trying to shift my thinking from “the material” to “the demonstrable, assessable learning outcomes” (cf. The Myth of Coverage).

Once I can articulate what I would like my students to be able to do after the class, it’s generally relatively easy to invent a few good clicker questions. I just formulate a question asking them to do that (in a particular context), and then much of the class activity is me helping them struggle through the process as they learn how. (This is the principle we’ve called “Question-Driven Instruction”, as articulated in Beatty & Gerace 2009 and elsewhere.)

Which all means that when someone says “Creating good clicker questions is hard”, I’m now inclined to hear that as “Thinking in terms of demonstrable student learning outcomes rather than topic coverage is hard.” And I agree. I also think it’s one of the many desperately needed shifts to how we conceive of this whole enterprise we call organized schooling.

I’m not saying that this is the only difficult aspect of creating good questions, but it’s definitely key for me. I’m curious what others think. If you’ve taught with a classroom response system, what do you think? Does that ring true? Do you have any similar or conflicting experiences to share? Comments are open…

Ask an educational researcher for his “teaching philosophy” and you’re likely to get a puzzled look and a long pause. These can be interpreted as “How do I condense years of research, literature reading, and theoretical development into a short answer?”

grounding

My philosophy of teaching draws from several research and philosophical traditions, as well as from the teaching experiences of myself and my colleagues. First and foremost, I am a constructivist (von Glasersfeld, 2007; Peschl, 2006). That term means many things to many people, but to me it means simply that knowledge and understanding cannot be “transmitted” between people; it must be constructed over time by each individual. In other words, learning is a deliberate process of sense-making that inevitably includes times of confusion, struggle, and reconciliation of difficulties. This relatively simple recognition has deep implications for instruction.

One implication is that communication cannot be taken for granted. All communication involves the sending of symbols that have no inherent meaning; meaning is intended by the sender and inferred by the recipient, and what meaning the recipient infers depends on his or her pre-existing expectations, assumptions, model of the sender, knowledge, and so on. As a teacher, that means I cannot presume that my spectacularly clear explanations communicate to the students what I intend them to. I need to model their interpretations as they model my intentions, and I need to “close the loop”, asking them to communicate back to me what they think they understood.

Another implication is that I do not “teach” so much as engineer a productive environment and set of stimuli for students to learn within, and provide coaching as they do so. (Note that this does not mean that lecture, or direct explanation, is always bad. Sometimes it is the appropriate stimuli to provide; nevertheless, I must remember that such lectures or direct explanations are not simply absorbed, understood, and immediately ready for future use.) Vygotsky’s notion of the zone of proximal development (Vygotsky, 1978) — that productive learning occurs within the space of challenges that students can succeed at with scaffolding, but not alone — suggests that I must continually tune the learning environment to students’ evolving capacities.

A third implication of constructivism is that students do not enter my classroom as blank slates; pre-existing knowledge, perceptions, perspectives, and experiences shape the understandings they construct in response to the environment and stimuli I provide. Thus, attempting to model their initial state, and track its subsequent evolution, is as important a component of my teaching job as designing my instruction.

Out of this perspective has grown the conceptual change tradition of educational research (Scott, Asoki & Leach, 2007), which studies the mechanisms by which students’ understanding of concepts evolves, the role of “misconceptions” in learning, and the like. More recent research, in what might be termed the knowledge in pieces tradition (Scherr, 2007), suggests that attending to what knowledge elements students have access to, and what contextual elements help to activate them, is more productive than considering what knowledge (or misconceptions) they “possess” (Redish, 2003; Hammer, Elby, Scherr & Redish, 2004; Dufresne, Thaden-Koch, Gerace & Leonard, 2005).

I am not just a constructivist, but a social constructivist informed by the sociocultural tradition of educational research (Carlsen, 2007; Mortimer & Scott, 2003). I see social interaction as essential to the internal knowledge-construction process, including student-to-student interactions as well as instructor-to-student ones. As Vygotsky observed (1987), the tools students use for internal cogitation are appropriated from social interactions. This implies that the classroom should be a place for exhibiting and exploring modes of thinking and argument, where students can see the process of “thinking science” modeled and where they can try it out themselves. Also, as Bakhtin observed (summarized in Wertsch, 1991, pp. 93-118), learning science largely means learning the social language of science (including conventions for thought and argument as well as vocabulary and grammar), and students must practice speaking a language to develop fluency. Thus, the classroom should be a place for students to practice “talking science”, with enough scaffolding from me to help them along, but not so much that I do the talking instead of them. In the very act of struggling to articulate their fuzzy thinking, students clarify their understanding of what they know, identify what they don’t, and often reach insights.

This has strong implications for what should occur in my classrooms. I do not see the classroom as a place for the dissemination of declarative content knowledge or the exhibition of proofs; those are more efficiently done through textbooks, multimedia, or other online resources. My classroom should be a place for dialogue and interaction, for exploration and confrontation and resolution. (In a large lecture hall, this is greatly facilitated by use of a classroom response system.)

My outlook is also shaped by the literature on student motivation and self-regulation (Koballa & Glynn, 2007; Wilson, 2006), and on the significance of students’ epistemological framing of the learning activities they engage in (Hammer, 1996; Hammer & Elby, 2003). Students are not black boxes, to whom instructional stimuli are applied and learning results; how they engage in learning activities matters tremendously, and as an instructor I must probe, model, monitor, and seek to influence that.

principles

Over time, I have distilled the practical implications for these (and other) pedagogical positions and educational research findings into four principles to guide instruction. These principles form the core of the “technology-enhanced formative assessment” (TEFA) pedagogy that my colleagues and I promote through in-service teacher professional development, and I would adhere to them in my own teaching.

The first principle is “Motivate and focus student learning with question-driven instruction.” This means posing tough, rich, meaty, often messy questions to students in order to contextualize and motivate subsequent learning, and often in order to catalyze or precipitate learning. It is grounded in the conceptual change tradition. It is motivated by an understanding that students perceive, process, and store information differently in response to a need, and that they “get” ideas by wrestling with the application of those ideas (Bransford et al., 1999, p. 139).

The second principle is “Develop students’ understanding and scientific fluency with dialogical discourse.” This means engaging students in discussion in which multiple ideas and ways of thinking are explored and contrasted, and in which students articulate and explore their own thinking. It is grounded in the sociocultural tradition.

The third principle is “Optimize teaching and students’ learning with formative assessment.” This means making students’ knowledge and thinking visible in order to adjust and optimize subsequent learning and teaching. It is motivated by an understanding that effective instruction requires detailed and current information about the specific students being taught, and that effective learning requires accurate self-knowledge (Wiliam, 2007). According to a seminal literature review by Paul Black and Dylan Wiliam (1998), “innovations” involving formative assessment produce learning gains that are among the largest ever found for educational interventions.

The fourth principle is “Help students cooperate in the learning process and develop metacognitive skills with meta-level communication.” This means communicating about communication, about cognition, about learning, and about the purposes of instructional experiences. It is grounded in literature on student motivation and self-regulation. It is motivated by an understanding that learning works better when students frame their participation appropriately and understand what they are supposed to be paying attention to.

I do not consider these four principles to be independent and arbitrary beliefs; they interlock and reinforce each other in a highly synergistic way. This can be seen in the way they are enacted in the TEFA “question cycle” — one specific way out of many of realizing the principles — which has been described elsewhere (Dufresne et al., 1996; Beatty, Leonard, Gerace & Dufresne, 2006).

references

Beatty, I. D., Leonard, W. J., Gerace, W. J., and Dufresne, R. J. (2006). Question driven instruction: Teaching science (well) with an audience response system. In Banks, D. A., editor, Audience Response Systems in Higher Education: Applications and Cases. Idea Group Inc., Hershey, PA.

Black, P. and Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1):7-74.

Bransford, J. D., Brown, A. L., and Cocking, R. R. (1999). How People Learn: Brain, Mind, Experience, and School. National Academy Press, Washington, D.C.

Carlsen, W. S. (2007). Language and science learning. In Abell, S. K. and Lederman, N. G., editors, Handbook of Research on Science Education, chapter 3, pages 57-74. Lawrence Erlbaum Associates, Mahwah, NJ.

Dufresne, R. J., Gerace, W. J., Leonard, W. J., Mestre, J. P., and Wenk, L. (1996). Classtalk: A classroom communication system for active learning. Journal of Computing in Higher Education, 7:3-47.

Dufresne, R. J., Thaden-Koch, T., Gerace, W. J., and Leonard, W. J. (2005). Knowledge representation and coordination in the transfer process. In Mestre, J. P., editor, Transfer of Learning from a Modern Multidisciplinary Perspective, chapter 5, pages 89-119. Information Age Publishing.

Hammer, D. (1996). More than misconceptions: Multiple perspectives on student knowledge and reasoning, and an appropriate role for education research. American Journal of Physics, 64:1316-1325.

Hammer, D. and Elby, A. (2003). Tapping epistemological resources for learning physics. Journal of Learning Sciences, 12:53-90.

Hammer, D., Elby, A., Scherr, R. E., and Redish, E. F. (2004). Resources, framing, and transfer. In Mestre, J. P., editor, Transfer of Learning: Research and Perspective. Information Age Publishing, Greenwich, CT.

Koballa, T. R. and Glynn, S. M. (2007). Attitudinal and motivational constructs in science learning. In Abell, S. K. and Lederman, N. G., editors, Handbook of Research on Science Education, chapter 4, pages 75-102. Lawrence Erlbaum Associates, Mahwah, NJ.

Mortimer, E. F. and Scott, P. H. (2003). Meaning Making in Secondary Science Classrooms. Open University Press.

Peschl, M. F. (2006). Modes of knowing and modes of coming to know: Knowledge creation and co-construction as socio-epistemological engineering in educational processes. Constructivist Foundations, 1(3):111-123.

Redish, E. F. (2003). A theoretical framework for physics education research: Modeling student thinking. In Vicentinni, M. and Redish, E. F., editors, Proceedings of the Varenna Summer School, “Enrico Fermi” Course CLVI. IOS Press, Amsterdam.

Scherr, R. E. (2007). Modeling student thinking: An example from special relativity. American Journal of Physics, 75(3):272-280.

Scott, P., Asoki, H., and Leach, J. (2007). Student conceptions and conceptual learning in science. In Abell, S. K. and Lederman, N. G., editors, Handbook of Research on Science Education, chapter 2, pages 31-56. Lawrence Erlbaum Associates, Mahwah, NJ.

von Glasersfeld, E. (2007). Key Works in Radical Constructivism. Sense Publisherss.

Vygotsky, L. S. (1978). The development of higher psychological processes. Harvard University Press.

Vygotsky, L. S. (1987). Thinking and speech. In Rieber, R. W. and Carton, A. S., editors, The Collected Works of L. S. Vygotsky. Plenum Press.

Wertsch, J. V. (1991). Voices of the Mind: A Sociocultural Approach to Mediated Action. Harvard University Press.

Wiliam, D. (2007). Keeping learning on track: Classroom assessment and the regulation of learning. In Lester, F. K., editor, Second Handbook of Mathematics Teaching and Learning, pages 1051-1098. Information Age Publishing, Greenwich, CT.

Wilson, T. D. (2006). The power of social psychological interventions. Science, 313:1251-1252.

I think that’s one of the reasons I like travel, especially adventurous travel to exotic places and cultures and conditions. Drop me into a totally unfamiliar environment, and I’m happy as a bug trying to figure out how to function. It also might explain why I’ve gone through a whole string of hobbies and enthusiasms and sports and professional interests: once the terrain gets too well mapped, so to speak, the appeal fades. Some people like exercising skill and expertise; I like acquiring it.

I may be somewhat extreme in this regard, but I doubt that I’m wired completely differently from most of the human race. I suspect that we learn most efficiently, and are most captivated by the learning, when we’re tossed into the deep end and have to figure out in a hurry which end is up.

In my experiences as a student, a teacher, an educational researcher, and a teacher of teachers, one of the things I’ve noticed is how reluctant most teachers (myself included) are to leave students behind. We conceptualize instruction as leading students along a carefully-engineered path to understanding, one step at a time; any student who stumbles, strays, or straggles and gets left behind will be lost. Thus, being conscientious of our responsibility to all students, we put great effort into ensuring that every one (or at least every one who tries) is with us for the whole journey.

And this, I think, has disastrous consequences. To prevent any from being left behind, we must keep all together in a tight cluster that moves more slowly than most need. Which means that we are nowhere near saturating most students’ capacity to absorb new ideas. Which means that many students get bored, and turn their excess capacity towards “off-topic” matters, such as side conversations, crossword puzzles, daydreaming, or social posturing. And which also means that few students learn as much or as rapidly as they are capable. Perhaps more insidiously, it means that few students have the opportunity to develop the intellectual skills essential to sense-making in a fast-and-furious environment.

At the root of all this is a fundamental misconception about pedagogy. Above, I said:

We conceptualize instruction as leading students along a carefully-engineered path to understanding, one step at a time;

As soon as we think this way, we’ve lost. That’s the “transmissionist” view of instruction, the idea that we transmit knowledge to students. That idea has been pretty thoroughly discredited in educational research circles, in favor of the constructivist view that students must construct their understanding through a sustained and effortful sense-making process. Knowledge isn’t a set or sequence of facts that can be presented in a logically optimal order; it’s a messy, complex, massively interlinked network of ideas and connections and perspectives and ways of thinking that can only be fully appreciated through extensive and repeated revisiting and re-contemplation. It’s not linear. As Jay Lemke observes (Lemke 1990, p.17),

In fact, it can be difficult or impossible to teach a thematic pattern one piece at a time because it often takes a mastery of the whole pattern before any of its parts seem to make sense. It is not just in science that we find concepts that can only be fully understood in terms of one another: Each piece of the puzzle makes sense only if you already have all the other pieces. This is one of the fundamental problems of science teaching, and indeed of teaching and communication generally…

What is the alternative? Throw students into the deep end. Engineer a rich, thorny, messy, meaty problem or question for them to wrestle with, dump on some ideas and tools that they haven’t yet mastered, and then let them struggle. Scaffold and coach, yes, but don’t try to lead them through. And definitely don’t try to force all students to follow the same path to comprehension. (This is, in essence, what my colleagues and I at UMPERG call Question-Driven Instruction.)

And maybe, if we can bring ourselves to do this, our students will thrive on school the way that I thrive on travel.

References

Lemke, Jay L. (1990). Talking Science: Language, Learning, and Values. Ablex Publishing, Westport CT. ISBN 0-89391-566-1. (Amazon)