At the AIM sponsored workshop Interactive Assessments in Open Source Textbooks, a group of participants had a lively discussion about how to include effective assessments in textbooks, especially formative assessments that occur in the middle of a section to engage the reader. Here are a few things I took away from the conversation.
The context was mathematics textbooks in general, at the level of developmental college mathematics and above. Below I refer to “students”, “readers”, and “learners” mostly interchangeably; the primary application we have in mind is a textbook assigned to a class of students, although independent learners should benefit from these features as well.
Formative assessments inside a textbook: the potential
There are a few different goals we might have in asking readers of a book to complete inline assessments.
- Correct misconceptions early.
- Help students assess their understanding of a topic.
- Provide an opportunity for students to practice solving a type of problem they just read about.
- Help students build their mathematical intuition.
- Help students build their metacognitive awareness.
- Point out new ideas; allow students to notice and wonder.
- Give students ownership over their learning to grow their identity as mathematicians.
- Encourage students to read actively, and more generally teach students how to become more effective learners of mathematics.
- Incentivize actually spending time with the textbook.
One thing that these assessments are not intended to accomplish is to punish students for not learning new material fast enough. For one, this means that the assessments should be graded mostly for completeness (and perhaps some appropriate level of effort) or even not at all. Further, we must be careful to avoid asking questions that students repeatedly get wrong: we are not trying to catch students, but rather to encourage them to grow.
Potential pitfalls
We must be careful not to include interactive assessments just because they are a shiny new feature that can be added. Having too many assessments can distract a reader from the larger points of the text, or frustrate them to the point that they do not read the entire section. Worse, formative assessments can do harm to the reader’s developing understanding by leaving the reader with false or misleading impressions of content or their level of understanding.
For example, a reasonable way to assess a student’s conceptual understanding of a topic is to present a simple true/false question. This is appealing because it offers the opportunity for immediate feedback, is easy to write, and can address a very precise unit of content. However, while such assessments might be appropriate for a summative assessment, when the learner’s concept is still forming, showing them false information has the potential for reinforcing a false belief. Suppose we ask, “True or False: \(\sqrt{a^2+b^2} = a+b\)”. Even if we have feedback that this is indeed false, a student might reasonably remember seeing the (false) equation in the textbook and forget the half that says it is false.
Here is an alternative way we might interact with the student to address the same topic. First, ask them to evaluate \(\sqrt{a^2+b^2}\) when \(a = 3\) and \(b=4\). Second, as them to evaluate \(a+b\) when \(a = 3\) and \(b = 4\). Finally, ask them what conclusions they can draw about the relationship between \(\sqrt{a^2+b^2}\) and \(a+b\): are they always equal or are they sometimes not equal? (This can be a multiple choice question).
The previous example illustrates how a formative assessment can be used to develop a students understanding rather than just assess it. The series of questions also demonstrate that a good formative assessment inside a textbook can do more than one thing at a time. In this case, the question gives the student practice with evaluating expressions, points out an important misconception that many students have, and models good mathematical exploration.
Purpose
Why do we have an assessment at this point in the book? Is it the right assessment? We must constantly be asking ourselves what the purpose of an assessment is. It is not there just to give the reader a task.
One way to ensure we are asking the right questions of our readers is to have a clear set of learning objectives for the section of the textbook. I personally find this challenging and not much fun. Mostly because it is hard. I would much rather be a story teller and just share my view of the mathematics I’m writing about. I’d much rather write a very clever and (in my mind) fun puzzle of a problem. But I’ll also admit that I really should be asking myself constantly, “what should students be getting out of this?”
What to ask
Certainly, a formative assessment can be as simple as giving the reader an opportunity to practice a problem analogous to a just-worked example. There are situations where this is appropriate. However, there are more conceptual questions that can lead to a student deepening their understanding. For example, we might ask a student to apply a general principle in a specific case, or to infer a general principle from a specific case.
And then there are open-ended free response questions. These seem like the perfect sort of assessment for this situation. For one, they give students practice in writing about mathematics. What are the alternatives for addressing a conceptual learning objective? Multiple choice questions make a student choose from a list. This might be appropriate some times, and we did agree that Parsons problems, where the student must arrange statements in a particular order, can be very powerful. But for students to grow as mathematical thinkers, they need the space to express their own ideas.
The problem of course is that there is no easy way to give feedback for students for open-ended questions. Even if an instructor has time to respond to each student, this will happen much later. So what can we do?
Scaffolded notetaking
One purpose of a formative assessment in a textbook is to encourage the student to take a moment to think about their understanding of what they just read. Asking a student to write about what they just read seems like a reasonable way to do this. So ask that! The book with keep the student’s response, and might even repeat it back to them later in the book (“earlier you were asked to describe the big idea in this theorem; here is what you said”) before asking them to respond again.
Many students do not know how to read a mathematics textbook. Many students do not know how to take notes. We can use our textbooks, with a built-in place to take notes, to teach students how to do both of these.
Open questions
There is a lot still to investigate. Some of this will become easier as we all start writing assessments inside our textbooks and get feedback from students and instructors. We suggest that authors use the OER Authors google group to share ideas and ask for suggestions.
I will also try to keep a running list of larger questions that need to be considered here. So far, there is just this one:
- Where do students do their work? For a computational problem, should the textbook provide space to write notes?