There are rules for what is required to pass, get a C, get a B, get an A. Roughly speaking a student who gets “Meets Expectations” on 3/4ths of the standards will pass with a D. “Meets Expectations” on every category is a C. To get a B they have to get “Exceeds Expectations” on 3/4ths of the standards (as well as Meets Expectation on the others). To get an A they have to have “Exceptional Performance” in half the categories.

There are some standards, like the constant velocity particle model, that I am not sure it’s possible to get “Exceptional Performance”. But it’s not necessary to get this mark on every standard to get an A.

Thanks for the updates, I like to see how this is going for you and how you are handling the various challenges. This is partly in response to your last post, and partly this one.

I read a blog post shortly before the semester started that spelled out the difference between formative and summative assessment (http://joshg.wordpress.com/2010/07/15/summa-wha-defining-our-assessment-buzzwords/). This prompted me to create two separate lists of standards, one which is formative and one which is summative — of course, they overlap. The formative list includes everything that can be learned in the class. The summative list includes only the things that they will impact the grade that goes on their transcript.

This way I can have as many formative standards as I want without worrying about overloading myself. They can be fine grained enough to where the assessment is almost binary. I can use this list to give the students formative feedback, e.g. You’re not able to solve Newton’s 2nd law problems because you aren’t correctly resolving vectors into components (even though there is no summative standards on vectors).

I have a weekly quiz and we’ve had 3 so far. There are physics problems on the quiz and the students are not prompted as to which standards are relevant in each problem. Frankly, that would be like giving them the answer (which may not be true for a different set of standards).

One of the things students are learning on these quizzes is that the right answer doesn’t matter. Just because you calculated the right numerical answer to the question doesn’t mean you demonstrated anything on the rubric for any of the standards. By the 3rd quiz I have some students who get it and some who still don’t. Students now start a problem by identifying the relevant standard(s) and then continue by applying those standards. (The standards are generally models, so if they do this right, equation hunting is eliminated.)

One issue that I have is something you mentioned in your last post. All standards are weighted equally in the final grade. This might be a mistake as some are probably more important than others. I recently had a thought that perhaps a good weighting between standards could be achieved naturally through a mind-map (which I just learned about from Andy ).

At the end of the semester, the students make a mind-map and the most highly connected nodes get higher weighting in the final grade. You can think of whatever meaningful function of the number of edges on the node you want. A side effect is that it is now impossible (or extremely difficult) for a student to even try to estimate their final grade during the semester. I suppose this might be good or bad, but one positive effect is that the students should just worry about doing well on the standards and not worry about their grades.

The faster pace is one difference for us postsecondary folks; the fact that we’ve got a smaller ratio of in-class time to (expected) out-of-class time is also big. It means that class time is very precious, and giving up any to assessment is difficult. I really struggle to assess enough to carry off SBG.

And I despise the weed-out mentality. Yeah, some of the students who come my way aren’t ready for physics (yet?), but many who would normally be “weeded out” can in fact be helped to learn and adjust and not just survive the course, but end up as much stronger learners in the future. I see it as part of my job to figure out *why* they “can’t do physics”, and then to help them change that.

I really like the way Brian Frank approached the idea of the more-encompassing standards by making synthesis standards that go with each unit: http://teachbrianteach.wordpress.com/2011/12/13/baby-sbg-jump/

I suppose one approach is to have assessed and non-assessd standards (whether pre-advertised as such or not). Another is to have “sub-standards” explicitly broken out for some or all standards, but to keep assessment at the main-standard level rather than the sub-standard level.

I still worry that the logical end of assessing higher-level, more-encompassing standards is that we’ll end up with “topics”: “You got a 3/4 on the kinematics standard, and a 2/4 on the Newton’s Laws standard.”

I had put in little tags like “s03.4″ at the end of each problem or problem-part to indicate which standards they were assessing, but some of the parts addressed more than one standard, and without the master standards list to look up the codes on I’m sure the students wouldn’t remember what the various standards were, and putting in the entire text of the standards seemed way too verbose, and anyway students should simply focus on answering the question… so I took the codes out.

Instead, I made up a grading form that I’ll fill out as I grade each exam, and then staple to the end to return to students. It’s got a list of the questions and their parts, with the relevant standard (code) next to it, and then a space for me to put the 0-4 mastery code. If more than one standard is relevant, I’ve got two lines for that part, one for each standard. If multiple parts work together to assess one question, I simply list them on one line, e.g. “1 abc”.

We’ll see whether that keeps SBG scoring from becoming a nightmare!

Ian, how have you set up your midterm? Have you chosen to clearly indicated to them the standard being assessed for each question or is that correlation of question to standard something that they will see after the midterm?

It seems to me that the more fine-grained the standards, the more helpful they are to students in directing their attention to what specifically they should be trying to learn. (Granted, we’d also probably need “integrative” standards like “I can choose an appropriate integration technique for a given integral.”) The counterbalance to that is that assessment and reassessment can become impractical for us.

(It probably won’t surprise too many experienced SBGers that I am, at this very moment, starting to regret having committed myself to as many standards as I have…)

What a lovely question you pose. I would say the difference isn’t size but language. “Chapter 1″ is of course the worst possible language because it communicates nothing of substance (and displays robotic planning on the part of the instructor). That would be “Chapter-based grading,” which surely goes on in some classrooms.

Something such as “Evaluate integrals” is better, but still awfully non-specific. That’s a topic (I think).

I have settled on breaking that into two pieces and specifying (admittedly in a vagueish sort of way) “challenging” integrals. Prerequisite to that is being able to evaluate simple integrals.

The next step would be to break everything out into the various integration techniques. “Evaluate integrals by u-substitution”, “Evalutate integrals by partial fraction decomposition”, etc. Lots of math folks go that route, and they end up with 80 or so standards. For me, that’s too fragmented. I’m thinking I can manage a dozen or so.

So for me there’s a big difference in the language. A standard ought to tell students what they can DO. A topic tells them the domain in which they are studying. Calc I has three topics-limits, derivatives and integrals. But that doesn’t tell students anything about what they’re supposed to be able to do.

Orals are optional, but will be necessary for anyone who wants to do a large number of reassessment or wants to get a large number of “Exceptional Performance” scores (i.e., get an “A”, or for some a “B” in the class). I think it will be possible for an average student to get a “C” in the class without doing oral assessments. I think every student would be foolish not to at least give them a shot, and I will counsel them about this.

Thanks,
:Ian

I’m also teaching Physics I this semester and using SBG. I taught Physics I last semester with traditional grading and was disappointed in two things: 1) students didn’t connect getting a question/problem wrong with what physics topic they weren’t understanding, 2) Students improved after the test and there was no good way to re-assess.

I also had a tough time making a good list of standards. When I wound up with a gigantic list, I realized there was no way I could manage with that many (and my class has a max 24 students). Eventually I decided to make two lists, essentially “formative grading categories” and “summative grading categories”.

The formative list has everything on it and borders on obnoxiously long and detailed. This list allows me to focus a student on a detailed task that they either do or do not get. So students will sometimes get feedback on a formative category even though they never get a grade for it.

The summative list contains everything they will get a score for that affects their final grade. It is essentially a subset of the formative list, but the topics are of the broader variety and there are only about 48 categories.

For example, vectors only show up in the formative list, not the summative list. They need to know, understand, and have skills in vectors in order to do many of the things they will get grades for, but their final grade does not directly reflect vectors.

To give a basic idea, the summative categories that I chose are based on what’s in Serway’s book. There are 12 “analysis models” (his term for an abstract model that can be applied to solve concrete problems), 17 “physical models” (these are things like free-fall, Hooke’s law springs, friction, etc), 5 “Physical Laws” (Newton’s laws, Conservation laws), 12 lab categories, and 2 “whole semester” categories (units, written/oral communication).

The Rubric requires being able to recognize when a particular model is applicable to a given physical situation and make connections between different categories in order to get a score of “Exceeds Expectations” or “Exceptional Performance”.

That’s my stab at SBG for Physics I. I’ll be interested to keep up with how it goes for you.

Thanks for sharing this and future updates. I’ll watch your progress with great interest. Like you, I sat in Andy R’s PERC talk last summer, and felt the light bulb go on listening to what he was talking about in his dynamics class.

Our intro physics course contains ~200 students and a mix of majors and non-majors (but all with calculus) in the same cohort. Even without taking the plunge to use standards for grading, I am finding it’s a really useful way to look closely at what we as instructors feel is important for students to master in a course. And you’re right about there being a lot of it in Physics 1!

I really like your point about the number of standards reflecting the importance of a topic. If I look at the learning goals I have developed for my own courses, there is a definite slow transition from the beginning of the course, where each little idea or skill gets its own, to the end of the course, where the standards become (as you say) more holistic. Unpacking those standards or learning goals for the later topics into the same granularity as the earlier ones does seem like an effortful task.

I also like the idea of having a healthy number of “above and beyond” standards because it allows for a system where students can get full marks on a regular standard for being correct, and there doesn’t always have to be room for them to get that “4″ by going above and beyond.

Topic: Evaluate integrals
Standard 1: Evaluate challenging definite integrals exactly
Standard 2: Evaluate challenging definite integrals by approximation

Embedded in Standard 1 is a whole mess of integration techniques. So in assessing that, I’ll throw parts, partial fractions, etc. at them on different assessments. If they can do some but not all of these, that’ll show up as a 2 somewhere along the way.

That’s my approach. Hope it provides food for thought.

Thanks for sharing. Keep us posted, OK?

Obligatory shout out to Jason Buell here.

Since you have some really specific skills necessary to understanding the concepts, I wonder if you can kind of bundle them. You could give students the full list of big goals with each individual skill bundled within them(so they know what small skills they will need to be able to do in order to comprehend the larger picture), but only assess and track the bigger ones. That would cut down on the number of standards in your gradebook and the number of specific reassessments. But, at the same time, if a student is struggling with a big concept, they know which specific skills to focus on.

I hope you include something in your syllabus about the big picture, even as you give them a detailed skills list, because I think it’s so important that students are reminded of it and know that you value it.

Thanks for sharing your process Ian!

One thing I worry about with fine-grained standards is that they slice-and-dice the material up into chunks and in the process lose the bigger-picture, holistic aspects. I’m thinking that perhaps having some whole-game standards as well as fine-grained ones might be optimal.

I’m actually leaning a bit that way in my current course, with standards like “I can use angular kinematics with the uniform and accelerated circular motion models to analyze physical situations” in addition to “I can relate angular variables (such as period, frequency, arc length, angular position, angular velocity, and angular acceleration) to each other and to linear variables.”

Yes, it turns out I did get the job, although it was a complicated scenario (cluster hire) so I’m not sure you can attribute too much to the teaching statement itself. No, I haven’t had any responses posted to the blog yet. (BTW, I’m currently rethinking my teaching philosophy… Not because I’m changing my mind about anything I said here, but because my research, experiences, and thinking over the past couple of years have taken me even farther.)

Good luck with your job search!
:Ian

This is one point where I may rubricize: Let’s say earning a “3″ in the standard means being able to execute it reliably, but earning a “4″ means also being able to recognize when to use it. Perhaps that’s a distinction that could apply across all standards?

Thanks for the response. Right now in my course I am mostly grappling with the fact that students may know a long list of things (kinematics, Newton’s laws, energy and momentum conservation, circular motion, etc.), but if I present them a problem (a physical scenario where they are asked to evaluate something) and ask them to tell me what physics principles, laws, ideas, etc. are important they have a difficult time giving a correct and/or confident answer. Often once they are prompted about what physics applies they can solve the problem. But to my mind, relating a real physical system to physics principles IS doing physics.

This is the reason I’m grabbing on to Serway’s short list of “analysis models”. I want to give the students a concrete choice — you must describe the problem in terms of these models. I think they are coming around to the idea.

I should back up a little though, because I went back and re-read Serway and he does use the word ‘model’ to cover more things than I let on (e.g. a spring is modeled by Hooke’s law). It is the phrase “analysis model” that he is very strict about. He defines an “analysis model” loosely as any type of problem we have solved previously, covering the behavior of some physical entity or the interaction between an entity and the environment.

You have a number of standards of the form “Use X as a model to analyze real physical situations”. All of these standards are linked to each other because the student needs to be able to distinguish between which model is appropriate when they are given the physical situation. It seems like this could be done without introducing a new standard depending on what your assessments are like.

Caveat: This list of standards, and the particular vocabulary I chose to express them, was scribbled down on an old note pad late one evening, and then hastily typed into WordPress the next morning. Please don’t interpret things like my use of “model” as thoughtful, carefully-considered choices.

It sounds like Serway is at least partially following the “Modeling Instruction” curriculum’s lead on the use of the term model… though I say that with only flimsy second-hand knowledge of either. I don’t personally think of “constant acceleration” as a model, but rather as a special case or subclass of behavior. I tend to use the term model in one of two very different ways: (1) as a theoretical description of how some realm of the physical world operates, as in “the Newtonian model of motion”, “a model of friction”, or “Hooke’s law as a model of a spring”; or (2) as a simplified, abstracted description of a physical situation in order to analyze it, as in “an ideal block on a frictionless plane as a model of a car on a sloped road” or “a frictionless point particle traveling in a circular arc as a model of a roller-coaster car”. The examples you cite of my use of “model” tend to be the latter ones.

Perhaps I should avoid confusion over the meaning of the term “model” and instead refer to these things as something else… “Common problem types” comes to mind, but that creates the wrong frame (that physics is about learning to solve standard problem types, rather than about building models of real situations). Thoughts?

Cheers,
:ian

I am currently teaching Calc-Based Physics I and wrote up a list of standards as a tool for my students to see what they are supposed to be learning (and possibly integrate it with the grading next term). I am really glad you put up your list so that I can see what an experienced prof comes up with. Our lists are totally different, while at the same time essentially covering the same things.

I am using Serway and he is much stricter with the word ‘model’ than you are. In the chapters you are covering here Serway only has 3 models: 1. Constant velocity, 2. Constant acceleration, 3. Uniform circular motion. Serway doesn’t label Free-fall and inclined plane as models in their own right, they are both examples of the constant acceleration model; Projectile motion is not a model, it’s an application of the constant velocity model in one direction and the constant acceleration model in the other. Accelerated circular motion is a little more subtle, but Serway presents the tangential and radial accelerations and from there applies the other models to understand accelerated circular motion.

Serway continues to be stingy with the word ‘model’ later in the book. Here are the rest of his models for Mechanics: Newton’s 2nd law, Isolated/Non-isolated systems for energy and momentum, rigid object with constant angular acceleration, rigid object with net torque (Newton’s 2nd law again), Isolated/Non-isolated system for angular momentum, rigid objects in equilibrium.

I wonder how you feel about being this stingy with what we consider a model? Does Knight use the word ‘model’ the way you do?

I recall reading (I think from Knight) that one of the big differences in problem solving between students and experts is the organization of knowledge. For example, experts will lump together many different problems as “Newton’s 2nd law problems” while students see “incline plane, free-fall, static equilibrium, atwood machine, etc.” and think they need a different approach to all of them. The student is paralyzed when they see a new system they’ve never encountered before.

I am currently trying to use Serway’s limited list of models to try to overcome this. Whenever we do problem solving in class, students are in groups and required to list the appropriate analysis models and we go over them as a class before anyone can write down anything further in trying to solve the problem.

I think that the appropriate granularity of the standards goes hand in hand with your planned methods of assessment. What do you have in mind for assessments?

1. A large minority of this course’s students (and a majority of those going on to Physics 2) are physics majors, and I consider 7 and especially 8 to be important skills for them, and indeed for any science major. Last year I only briefly touched on Fermi estimation; and I would reluctantly drop it if I had to trim coverage. #8, however, is something I weave throughout, as a general habit of sanity-checking answers and even deducing the forms of expressions.

2. I hear you, and granularity is something I’m struggling with. However, I really don’t want to end up with a bewildering array of unitary standards. In addition, I try to teach the position/velocity/acceleration relationships as organic and interrelated, so I lean towards one encompassing standard.

3. Unit vector notation was intended as part of #16.

4. I mean that as an implied component of #17, but the standard was meant to be larger — more like “Can you actually figure out when you need to use vector stuff when solving physics problems, and apply the right bits in the right places?”

5. In theory, Calc I is a pre-requisite for Phys I here, although a nontrivial minority are taking it concurrently (with my explicit permission). I don’t mean to get very hairy with the calc, but I do want them to link to the basic ideas of derivative (slope) and integral (area under curve), and connect the graphical concepts to the basic calculus operations. (General note: I intend a few standards to be in there as “advanced” ones, to separate the A-level students from the rest. This is one of those.)

1. In actual practice would be so comprehensive? I’m thinking of items 7 and 8 in Chapter 1. Would you really try to hold students to that?

2. Item 5 in Chapter 1 seems like it needs to be broken down more, into position, displacement, velocity, acceleration.

3. In Chapter 3, I would probably add something about being able to correctly use unit-vector notation; students have lots of trouble with this at first.

4. For item 17, Chapter 3, do you mean something like “Correctly identify which physical quantities are vectors and which are scalars.”?

5. Do most of your students actually have calculus under their belts? Most of mine are just starting calculus, so items like 10 wouldn’t work in my class. We only use calculus conceptually (finding slopes and areas mostly), so I would use standards based around those.

Perhaps Ender would have had more fun playing World of Warcraft, though?

Thanks for compiling all of this. I conducted a study on the efficacy of clickers in high school sciences classes for my doctoral dissertation at the University of Oregon. For what it is worth, you were cited several times when discussing the importance of posing high quality questions when using this technology. In the process of the study, I created a monster in that all teachers in multiple departments want to use them. Your site is a great resource for our teachers and I appreciate the continued effort.

My understanding—based on the documentary, primarily, so caveat emptor—is that constructive interference of “normal” waves, according to the generally accepted linear model of ocean waves, is inadequate to explain the size and frequency of rogue waves. Basically, if that were the explanation, we ought to see rogue waves once every 10,000 years, rather than multiple times per year. So yes, I gather that the linear model is outdated.

(BTW, the “quantum physics” version isn’t actually predicated upon quantum physics; I believe what’s being suggested is that the mathematics of ocean waves is analogous to the mathematics of (certain portions of) quantum mechanics.)

Cheers,
:Ian

Thanks for compiling these, Ian!

Frankly, when I write for this blog I generally get the sense of shouting into the void — nobody listening — which isn’t very motivating. Catch-22: Minimal writing without readers, no readers without writing.

Thanks for noticing and asking, though!

Are you still maintaining this blog? Or just taking a hiatus?

Stephanie

I think a lot of instructors who start using clickers do so in order to gather feedback on what their students do or do not understand during lectures. Some have a vague notion of increasing interaction during class, but I think you’ve hit the nail on the head regarding the use of clickers to *engage* not just assess students. When you do so, the clicker question becomes a vehicle for encouraging the kinds of thinking we’d like our students to do.

Also, it would appear your comments are working again!

That is an excellant method, it shows that not all ‘truckers’ mind being overtaken, as the mojority in the UK seem to beep whenever this is done.

However, it has to be said that the majority of country lanes in the UK do not allow trucks through – and as such, they have prescribed routes which have both a slow and fast lane. So yellow line driving would not be needed over here.

Interesting post ! I will study this further.