I love it when two different ideas collide in my head, even when I’m not entirely sure whether the result is a beautiful synthesis or an ugly wreck.
In a conference paper I wrote exploring what we could learn about physics teaching by studing effective video games, I said:
To me, themes of grounding, exploration, assessment, and feedback suggest that we ponder how the process of learning physics can be re-conceived in such a way that students learn physics by encountering and exploring some experiential “terrain” in which physics ideas are manifest, receiving immediate, obvious, and natural feedback about their developing understanding and competencies. One key ingredient for making feedback immediate and apparent, in a manner that scales, might be the development of students’ self-assessment capacities in parallel with content learning. Students must learn to self-check and peer-check in the same way that practicing scientists do.
When I wrote that, I was vaguely envisioning possibilities such as having students accompany each problem solution they produced (on homework or an exam) with a detailed self-critique of the steps they were more and less confident about; including frequent peer critique of assignments; and so on.
I do a fair amount of computer programming (and used to do much more), and for several years I’ve practiced test-driven development. Loosely speaking, this is an approach to programming that advocates:
- Writing test code that automatically tests all desired aspects and features of the program one is developing;
- Organically growing a program in small steps, beginning with a version that functions correctly for a trivial subset of the desired purposes, and then gradually adding functionality a bit at a time, each time producing code that passes all tests (for the features so far implemented) before proceeding; and
- Writing the test code for a new feature (or for the successful repair of a newly discovered bug) before writing the code for the feature or making the bug repair, so that the test initially fails, and then the feature/fix is done when the test succeeds.
The test-first discipline is difficult to self-enforce, but I find it quite helpful for anything beyond the simplest of programs. For one thing, it’s the only realistic way I’ve found to reliably produce a thorough, complete, trustworthy safety net of test code. Having such a safety net allows me to freely tinker with working code to “refactor” it and make it better and more elegant. For another, having to think through the test cases up front helps me clarify exactly what the desired feature needs to do and how the code implementing it needs to be factored, which leads to fewer dead ends and “Oops, I need to do this a different way” moments. For a third, writing code so that all the logical pieces are independently testable forces me to break my program up into simple, loosely-coupled chunks, which is generally good practice.
For the last few days, I’ve been teaching myself to program in R in order to do some statistical analysis. The code I’ve been developing has started to get more complex, so yesterday I figured out how to set up a framework for automatically testing R code.
The collision occurred this morning, while rereading that conference paper to prepare for an upcoming talk. It occurred to me: “What if we treated building knowledge like building a computer program, and established self-tests for the things we wanted to learn before we learned them? What if we could get students to buy into this approach?”
What would that mean in practice? I’m not entirely sure. To develop test code, I have to figure out what “feature” I want to add to my program, and envision how the program will behave (as seen from the outside) when the feature is implemented. I’ll then write test code that tries to make the program exhibit that feature in a variety of circumstances, and squawks whenever the program’s behavior deviates from the expectations coded into the tests. Then I’ll run the test suite, which will of course squawk like crazy since I haven’t yet actually written the feature into the program. Only then do I begin engineering the feature into the program, testing as I go, until the test code gives me a thumbs-up. That’s when I know I’m done.
Actually, I’m not quite done then. The final step is to review the feature’s implementation, and see whether I can make it cleaner, faster, more elegant, or otherwise better. While so doing, I intermittently re-run the test suite to make sure I haven’t unwittingly introduced a bug.
Let’s think about this with “learning some physics” in place of “developing software”. My first step is to identify the “feature” I want to add, which is like identifying the physics I want to learn. Next, I have to specify that feature: How, precisely, will the program behave when the feature implemented? What will it do in various circumstances? In learning physics, this maps to specifying what the desired knowledge will allow one to do that one can’t currently do. I think students are often fairly fuzzy on that, and introducing this into students’ learning process could be a big win. I’m in favor of anythign that gets students to think about physics as “capacities to develop” rather than “stuff to know”.
Next comes the hard part: writing the test code. In the physics case, that means (the learner, not the instructor) setting up self-tests of some kind(s) that one can’t currently pass, but expects to be able to pass after learning the desired thing. That means operationalizing the specified capacities very concretely. This will likely be the hardest step: basically, making up test questions (or other assessments) before one has learned what the test will be testing. And it’s a bit of a trick, too: In the process of doing this, one is already beginning to do the learning. I find that as I write test code for a desired program feature, I’m already beginning to see the shape of the implementation code in my mind. I’m sure that somebody, somewhere, in some context has sagely observed that clarifying a problem is often the most important step in solving it.
Once the self-test is developed, go learn. Unike the vague and rudderless learning that students often do, however, guided by no objective more specific than “Get ready for a forthcoming but unknown exam”, one has very specific questions/challenges that one is seeking answers/solutions to, some thing I have called “question-driven instruction” elswehere. I firmly maintain that in real life, most of us learn something (like how to program in R) with a very specific objective in mind (like analyzing a particular data set), and that having a clear goal helps us learn more effectively and efficiently by providing direction and by giving us a way to structure and organize what we learn by its utility rather than by the chronology of a textbook or course.
The feature is implemented when all the tests in the test code are passed. Similarly, the student is done learning (that bit) when she can successfully complete the test tasks she originally laid out. Imagine the self-confidence that could arise from that!
Of course, it’s common in test-driven programming to discover bugs or incompletenesses in the program that one hadn’t anticipated and built into the tests, often when the program is tried in a new context. Similarly, a learner will no doubt discover weaknesses in her understanding, revealed by contexts and questions she had not originally contemplated. No worries: The appropriate thing to do is to immediately develop new tests that capture this weakness, such that the tests are currently failed but will be passed when the weakness has been satisfactorily remedied. This is an expected part of developing a program organically through test-driven development, not a failure of the system.
If you’re trying to envision this process playing out in a physics course, please keep in mind that that this is an iterative process, feature by feature, and that each iteration attacks a fairly small chunk of the overall problem. If you’re imagining asking students to “design self-tests that will let you know when you understand Newton’s laws”, you’re thinking of chunks that are way too large. A better example might (possibly) be “design self-tests that will let you know when you can predict an object’s position at any future time, given the object’s initial position, velocity, and (constant) acceleration.”
Might such a pedagogical approach be practical? I have to say “of course it is!” — at some level — because that’s the way most people learn most things, outside of school. When our learning is in response to a specific need, meeting the need is the test. If the need is complex, we break it down into compoents, and learn what we must to overcome them one at a time. Only in “school learning” do we try to pack knowledge into our heads because we’re told we should learn this stuff and that it will be useful eventually, and only in school do we depend on someone else to tell us whether we understand what we hope we do.
Would it be too slow, with all that test-inventing taking time away from the actual learning? Many people’s perception of test-driven software development is that it’s slow, because one often spends more time writing the test code than writing the code that actually implementats the features. However, practitioners of test-driven development also tell you that such time is not lost; it is invested, and recovered with dividends as the program gets increasingly complex, bugs appear that must be found and squashed, new features must be added on without breaking old ones, and the program must be checked out before being trusted. “I *think* this program works, because it seems to when I fiddle with it, and all the programming decisions I made seemed right at the time” is not a comfortable place to be.
Is this collision an ugly wreck or a beautiful synthesis? You decide.