R305A150456

The SERP Institute partnered with Temple University Professor Julie Booth and teachers from several school districts to develop and test “MathByExample”—a set of math assignments for 4th and 5th grade students strategically designed to target common misconceptions and errors. This IES-funded research and development project is rooted in partnership work that started over a decade ago aimed at improving student learning and reducing the achievement gap in Algebra 1. Drawing on the research literature, assignments were designed with correct and incorrect “worked examples”—problems that have solutions worked out and marked as right or wrong—alternating with problems to solve. Despite being “light-touch”, the Algebra assignments had a surprisingly powerful impact and have been downloaded worldwide. Results demonstrated even bigger improvements with students at the lowest end of the performance distribution, and student responses revealed that many misconceptions were rooted in earlier grades. Thus, the team shifted focus and applied the same approach to fourth and fifth grade mathematics content. Results from a randomized controlled trial in five districts show similarly positive results in upper elementary grades. But even more exciting might be the ramifications of making a fictitious student’s error the focal point: engaging students in analyzing and critiquing the reasoning of others—a goal of 21st century standards, learning from those errors, and, perhaps most importantly, changing the perception of making mistakes in math class. By strengthening students’ foundational mathematics understanding, teachers in higher-level mathematics courses will be freed to focus on the grade-level content, increasing the opportunity for more students to succeed in higher level mathematics.

## Allie Huyghe

Lead PresenterAssistant DIrector

Hello everyone! We hope you enjoy this video. Please share the materials with your colleagues! The assignments are freely downloadable from the MathByExample website: https://math.serpmedia.org/math_by_example/

To learn more about how this partnership work started, you can visit the AlgebraByExample website, linked here: https://math.serpmedia.org/algebra_by_example/the_r-d.html

Let us know if you have any questions!

## Susan Jo Russell

Hi Allie and Julie,

Thanks for your intriguing video--making errors a focal point to be learned from rather than avoided seems very promising. I was interested in your reference to the importance of critiquing the reasoning of others. Are students generally interacting with your materials individually or is there also collective discourse?

I was also curious about how you've thought about procedural versus conceptual thinking. You highlight the example of lining up (or misaligning) the decimal points in an addition problem. What have you learned about how students using your materials come to understand about

whythey line up the decimal points in this way, in other words, what are they learning about the magnitude of the numbers? Does estimating a reasonable result play a role? Does the work support only the standard carrying algorithm or does it also support other algorithms or strategies?Thanks so much.

## Allie Huyghe

Lead PresenterAssistant DIrector

Hello Susan,

Thanks for your message! We encourage teachers to use the assignments as they would any other assignment in their class. Some have students work on them independently, while others ask students to work with others. We find both to be effective. Teachers have generally noted a preference for students to complete the assignments in pairs and groups, since the self-explanation questions naturally prompt student-student discussions and can be a great starting point for mathematically-rich whole class discussions as well. Even if working independently, students are engaging in critiquing the reasoning of others since they are analyzing and responding to prompts about a fictitious student’s work.

While some of our assignments focus on procedural knowledge, we strive to mainly target conceptual understanding. In other words, we focus more on the “why,” rather than on the “what” or “how.” Some of our assignments do target magnitude and other do focus on estimating reasonable answers. Finally, we strive to support multiple strategies throughout the set of assignments. For instance, we demonstrate the standard algorithm, as well a various models/representations used in the range of CCSS-M aligned curricula our partner districts have adopted. To get a sense of the range of concepts we target with the assignments, it is helpful to look at the full set of assignments for each topic. You can check out the various assignments on the website: https://math.serpmedia.org/math_by_example/

## Denise Schultz

Hi Allie! Your work reminds me of one of my favorite math routines to do with students called "My Favorite No" where the teachers displays a common error made by either a real or fictious student and says something like "I love this mistake because we can learn so much from it" The teacher may ask what is right about the thinking and where the thinking goes wrong. Like your work, its meant to banish the stigma of a wrong answer in math class and to see it as an opportunity to learn from. I'm curious if your work includes collecting data on the affective side to learning mathematics.

## Allie Huyghe

Lead PresenterAssistant DIrector

Hi Denise! Thanks for your comment. You’re right, “My Favorite No” definitely seems like a similar strategy for banishing the stigma around errors and getting students to tune in to particular features of the problem and shift away from answer-getting! In terms of data collection, our work focused primarily on performance outcomes. However, we did include pre- and post- surveys, and included expectancy and value items, as well as items about the error climate of the classroom. These items asked students to indicate how errors are used in their math classrooms and how they feel about the use of errors. We anticipate that the MBE assignments may increase students’ perceptions of the functionality of errors as a learning tool in the classroom. These data are still being coded and analyzed.

Denise Schultz

## R. Bruce Mattingly

Very interesting work! I'd like to share it with my colleagues who train future elementary math teachers. I'm also wondering if you think this same approach would work at higher levels of mathematics, perhaps after some of those misconceptions have taken root?

## Allie Huyghe

Lead PresenterAssistant DIrector

Hi Bruce! It would be great if you shared this with your colleagues -- we think this would be a great tool, especially for new teachers, since it allows teachers to gain insights into students’ thinking about these key mathematical concepts. You can access the full set of assignments on the website: https://math.serpmedia.org/math_by_example/

As for higher-level mathematics, we actually started this work in Algebra 1, developing and testing AlgebraByExample. It was with AlgebraByExample that we found significant results for students’ procedural and conceptual understanding, with greater gains for students at the lower end of the performance distribution. The full set of these assignments can be found on the website: https://math.serpmedia.org/algebra_by_example/ For more information about the background of the project, please visit: https://math.serpmedia.org/algebra_by_example/the_r-d.html We have also applied for funding to expand to high school Geometry content–GeometryByExample.

## Karin Lange

Hi Bruce and Allie!

I was tangentially involved with the AlgebraByExample work a few years ago. I have recently taught undergraduate mathematics, and I have thought a lot about how these materials could apply in a higher education setting as well! I think it would be an excellent way to address the misconceptions that continue to appear in undergraduate math classrooms. I am so glad to hear that there are plans for GeometryByExample! This is great work!

## Mahtob Aqazade

Thank you for sharing your video! We also incorporate the use of worked examples in our research as part of our interventions with elementary students. I liked the way that you described how you focus on "why" rather than "what" and "how" in order to promote conceptual understanding. I am curious to learn more about other approaches that could help students engage more when analyzing worked examples, for example, using illustrations/models/graphs. I am also interested in learning the different or similar forms of a worked example when targeting a concept in geometry or measurement. Thanks!

## Allie Huyghe

Lead PresenterAssistant DIrector

Hello Mahtob, Thank you for your comment. Our assignments utilize a wide variety strategies, including models and graphs. The assignments also align to all CCSS-M standards for 4th and 5th grade, including geometry and measurement. To get a sense of the range of strategies and topics, it is helpful to look at the full set of assignments. You can check them out on the website: https://math.serpmedia.org/math_by_example/

In addition, prior to MathByExample, we designed a similar set of assignments focused on Algebra I. They can be accessed on the website: https://math.serpmedia.org/algebra_by_example/

For more information about the background of the project, please visit: https://math.serpmedia.org/algebra_by_example/the_r-d.html

Please let us know if this doesn't answer your question, we'd be happy to elaborate further!

Mahtob Aqazade

## Mahtob Aqazade

Thank you so much for sharing these resources!

## Beth Sappe

Hi Allie, Thanks for sharing the informative video. As you know, some teachers in my district have been using the Math By Example resources. What type of professional learning have you conducted with teachers that are part of the pilot? Are teachers guided to give students feedback or are they utilizing data to make decisions about future learning?

Can you share some of the positive results that have been surfaced so far?

## Allie Huyghe

Lead PresenterAssistant DIrector

Hi Beth! Great to hear from you here! In the original partnership work that led to the development of AlgebraByExample and then later MathByExample, one of the constraints district leaders put on the design of the approach was to not require professional development and training in order to adopt the materials, so we had to strategically design “light-touch” materials that were easy to adopt and fostered "learning-by-doing." And many teachers have reported exactly that -- they reported learning from these materials both in terms of research-based practices and gaining insights to how their students are thinking through the mathematics. So in the pilot conducted with both “ByExample” resources, no professional learning was conducted with teachers. We have, however, hosted some meetings with teachers who were part of the development team to review the theory behind worked examples, self-explanation, and dos and don’ts for creating your own worked examples, so the project team has discussed the possibility of creating professional learning sessions around these topics for dissemination as well. For example, my colleagues also authored “A Worked Example for Creating Worked Examples” that can be used as a resource, linked here: https://www.researchgate.net/publication/281066179_A_Worked_Example_for_Creating_Worked_Examples

As you mention, there is also the possibility of providing guidance to teachers on giving students feedback and on utilizing data to make decisions about future learning. In many cases, because this is meant as a supplemental resource and not a comprehensive curriculum, we’ve had pushback to add “one more thing”, since extensive professional learning and guidance is often a requirement of the core curriculum and is given priority by districts. So, to date, this hasn’t been on the radar, but we are certainly open to the possibility as we continue to explore how we can further support teachers and districts in adopting and implementing the assignments. I’ll definitely be in touch with you as we think more about it to see whether it aligns to what you are imagining would be helpful.

As for results, we are still combing through data and entering all correct/incorrect/completion data from the workbooks, and we won’t have final results until that work is done. But preliminary analyses are showing that even without taking individual assignment completion into account, 5th grade MathByExample students improved their procedural knowledge and conceptual understanding more than peers who only completed problems to solve (identical problem sets, but without the worked examples and prompts). I will send along more details once we have more refined results to report!

## Beth Sappe

Thanks for sharing Allie. I look forward to hearing from you soon. I am always open to finding ways to support teachers so when we touch base lets discuss that further.

## Kathryn Kozak

This is really interesting. I teach at a community college, and many of our students are in beginning algebra or below. Have you looked to see if this can be expanded to students at this level?

## Allie Huyghe

Lead PresenterAssistant DIrector

Hi Kathryn! SERP’s focus is on K-12, but we have heard of the AlgebraByExample materials being used at the community college level. If you decide to try out the materials with your students, please let us know how it goes! Perhaps there is also an opportunity to create additional worked example assignments that align to your content as well. I’ve pasted some resources below:

AlgebraByExample website: https://math.serpmedia.org/algebra_by_example/

“A Worked Example for Creating Worked Examples”: https://www.researchgate.net/publication/281066179_A_Worked_Example_for_Creating_Worked_Examples

## Lizhen Chen

Hi, Allie, this is a very intriguing video. You said the prompts (e.g., “What did Zach forget to do when multiplying 316 by 3?” and “Using estimation, explain how Zach should have known his answer was unreasonable.”) would encourage student-student discussion. I’m curious about students’ language use in their discussion. Did you have any control/ intervention (e.g., sub prompts) for students’ language use? Or you just facilitated, saying “What did Zach forget to do when multiplying 316 by 3?” and left the time for students till they finished and were ready to move to the next question, i.e., “Using estimation, explain how Zach should have known his answer was unreasonable.”

I like the idea of solving similar new problems after discussing worked examples to see how much conceptual and procedural knowledge students could get from the worked-example discussion. Are there any trends that you find about students’ success and difficulty in the learning transfer?

## Allie Huyghe

Lead PresenterAssistant DIrector

Lizhen - Thanks for your comment! The prompts on the page are actually intended to be written responses, but when done in pairs or groups, they naturally prompt discussion between students. Some students may not recognize what Zach forgot to do when multiplying 316 by 3 and different students may have different strategies for knowing whether an answer is unreasonable, for example. By having students complete the items in pairs or groups, it is not uncommon for a comparison of potential responses to lead to a discussion about the content of the example.

Our primary research focus was whether these assignments were more effective than traditional assignments with only problems to solve relating to students’ academic performance. We did not specifically study language use in discussions formally; only noting how the assignments were generally administered (e.g., independently, in pairs) and noting student utterances during observations to help inform revisions. It would certainly be an interesting extension of a future study, however, to look more carefully at the types of discussions students were having and how they varied with the different types of prompts as well!

Interestingly, for a dissertation, a colleague did an analysis of the formality of the language used in the AlgebraByExample written responses. We looked for differences in learning based on whether students used the formal language (i.e., coefficient) or informal language (i.e., number next to the x). We found that the type of language used did not affect students’ learning. As long as students attempted to explain their reasoning (regardless of the type of language used), they benefited from the AlgebraByExample assignments. See https://www.researchgate.net/publication/326714919_Precise_mathematics_communication_The_use_of_formal_and_informal_language for more details.

Further posting is closed as the showcase has ended.