NSF Awards: 1720930
"Improving Practice Together" (IPT) is a four-year partnership between three institutions to build a school district's capacity to teach scientific argumentation by developing sustainable teacher leadership.
Our Research Practice Partnership (RPP) is unusual because it involves three partner institutions:
- a professional learning team and an evaluation team from the Lawrence Hall of Science of the University of California at Berkeley,
- a research team from Stanford University,
- and a team of teachers and administrators from the Santa Clara Unified School District (SCUSD).
The Lawrence Hall of Science facilitates Professional Development (PD) activities for a first cohort of Santa Clara teachers who become teacher leaders later in the project. After participating in PD activities and implementing argumentation for a year, the first cohort participates in a Leadership Development Institute, to be able to co-facilitate PD activities with the Hall for a second cohort of teachers from Santa Clara. The teacher leaders will then facilitate the professional learning activities for a third cohort the following year, with less support from the Hall, then independently for more cohorts in the future
The research team investigates questions about the project’s processes and outcomes related to classroom learning and teaching, to teacher professional learning, and to district capacity building to conduct professional development. They are also studying the partnership work itself.
The SCUSD teacher leaders, the science teacher on special assignment, and district administrators participate in decision making about both research and professional development activities, to ensure that all structures and activities meet the district's needs. They consider structural changes, resources, and activities that are necessary to build the district's capacity and leadership to teach scientific argumentation.
All partners meet regularly to tackle challenges and to achieve project goals together.
The project has an important impact on the sustainable improvement of teacher practices because the SCUSD is a partner in the process of engineering a program that will work for them after the project is over. It also impacts student learning, as giving students the opportunity to argue for a particular idea is valuable in developing (a) their understanding of the idea and (b) their skills in communicating, listening, and engaging critically with their peers. Finally, the project provides a unique opportunity to analyze the benefits and challenges of working in a three-way partnership to develop teacher leadership in school districts implementing educational reforms.
Coralie Delhaye
Postdoctoral researcher
Welcome to Improving Practice Together, a three-way partnership to support discussion and argumentation in elementary science education.
We are in the middle of the second year of the partnership and are getting ready to run the teacher leadership academy.
We are happy to share about the strategies we use to support argumentation, the professional learning model, our measures of change in classroom discourse practices, and our research on the evolution of the partnership.
We hope that everyone who visits will comment on our work! How does it compare to your own experiences teaching and learning science?
In addition, we have some specific questions for folks doing similar work.
We look forward to discussing these questions with you!
Warmly,
Coralie, Emily W. and Emily R.
Sally Crissman
Senior Science Educator
I am very eager to follow your important project as it unfolds! The first thing I noticed with joy in your video was The Circle-a basic strategy! Discussion and argumentation are essential science practices but hard to pull off, especially whole class discussion and facilitation of productive small group work. Our curriculum development work, the Inquiry Project, relied on these exploration and meaning making strategies and ultimately we developed a parallel project Talk Science.
I’ll toss out one suggestion for preparing teacher leaders: make sure they have plenty of experience as a participant in a productive discussion and time to analyze the ingredients that made the discussion productive - or not. In my experience, not many teachers experienced science learning that included discourse so it isn’t in their bones when they start teaching.
I look forward to this discussion - a topic very close to my heart!
sally
Coralie Delhaye
Emily Weiss
PI Improving Practice Together
Hi Sally,
It's great to have you as a facilitator. We have really admired your work on the Talk Science project. We actually share some of your videos with our teachers during professional learning sessions and have pointed them to your resources as well. We agree that having teachers engage as learners, using the practice of argumentation themselves, is really important. It's been quite fundamental to our professional learning program, along with helping them think through how they can support students. We find that even a year into the program with our teacher leaders (who went through the same program they will be leading) we are still needing to have them engage in argumentation activities as learners because what they pull out of the experiences becomes more nuanced as they try argumentation in their own classrooms. They pick up on facilitation moves differently, e.g., helping students visually distinguish between data and evidence when they had never made this distinction before. I look forward to more conversation.
Best,
Emily
Sheila Homburger
Thank you for sharing this wonderful project! I recently worked on an education research project involving curriculum development for high school students, with a focus on argumentation from evidence. It's exciting to see a project that introduces these skills to younger students.
I'm curious about two things. One, what are your strategies for introducing the concept of argumentation to students, and what challenges are these strategies designed to navigate? And two, how are you measuring (or planning to measure) gains in students' argumentation skill? Related to part two, what does successful argumentation look like at for elementary students?
Sheila
Coralie Delhaye
Nicole Wong
Coralie Delhaye
Postdoctoral researcher
Hi Sheila,
Thank you for joining the discussion about this project and your very interesting questions.
The research team at Stanford has indeed developed an instrument that allows to rate the teacher and student types of practices that Emily described (teacher: "ask", "press", "link" ; student: "claim/explain", "co-construct", "critique").
Data: Videos of classroom science discussions.
Data collection: We collect three videos before the teachers participate in professional learning activities, to use as baseline data. We then collect three videos every spring and every fall for each year teachers participate in the project. We select the segment that contains the best science discussion in each video.
Data analysis: Each segment is analyzed individually by a pair of raters. They rate each rubric on a scale from 0 to 4, depending on how frequently they observe practices that fall into that category and on how proficient they are. For example, rating a segment as 4 for “Press” means that there is evidence that “The teacher consistently presses students in the class to provide evidence/reasoning for their own comments or the comments of another student. There are few, if any, instances, when the teacher missed opportunities to elicit deeper thinking by not asking students to provide evidence/explain reasoning”. Rating it as a 2 means that “The teacher rarely asks students in the class to provide evidence/reasoning AND/OR there are some/many examples of the teacher engaging in emerging versions of press. There are many instances of missed proficient use of the press, where the teacher needed to press and did not”. An “emerging version” of this practice is asking students for elaboration of a claim, without asking for evidence or reasoning.
Analysis reliability: To increase reliability, we train the raters, we ask them to meet regularly and discuss their rating in pairs to come to a consensus when they differ, and we organize rating "calibration" meetings where the whole group rates the same videos to make sure that we have satisfactory inter-rater reliability coefficients.
Are you interested in evaluating argumentation in your project as well? I saw that you are working on students’ understanding of evolution. That is a great topic to practice argumentation!
Best,
Coralie
Sheila Homburger
Sheila Homburger
Thanks for your response. These details are very helpful for understanding your process.
Best, Sheila
Emily Weiss
PI Improving Practice Together
Hi Sheila,
Thank you for your questions. I really had to think about my answer to question one (what are your strategies for introducing the concept of argumentation to students, and what challenges are these strategies designed to navigate?), and I may need to add to it later. I would say that one of the biggest challenges teachers have is just allowing for a classroom to become more student centered—shifting the culture so that students feel like they can be knowers and sense-makers and that the teachers trust them to get there. I'll get to that more in a minute. Generally, our teachers have been introducing argumentation as a way that scientists and engineers come to understand the world together or make decisions together. I think most of them focus on argumentation more as discussions that help with these processes and don't focus on the term argumentation so much. Within this idea of sense-making (e.g., explaining phenomena) and decision-making (e.g., the best way to conduct an investigation or solve a problem), the teachers makes clear to their students that there will often be disagreements and that this is normal and useful—that when we all share our thinking and evidence for that thinking, we all learn, especially when some of the new evidence changes our thinking. This being said, many of our teachers are still considering whether it's more productive to have students disagree with each other or with ideas. I think this is a great place for them to be because it means they're thinking about the essence of why they are having students engage in this practice. Allowing more open-ended discussions to happen in a classroom requires dramatic shifts in student and teacher expectations. Students may be uncomfortable sharing ideas until they think they are the "correct" idea for a variety of reasons. They may need to work on listening skills that require critical thinking, which goes beyond parroting back information. And teachers need to have approaches to support students in building this comfort and the skills to participate in discussions. For teachers this means a lot of letting go, but it also means finding the right balance of letting go and maintaining control of the direction of a discussion to keep it productive. Teachers spend a lot of time in the first year of this type of work figuring out where it's okay to step back and where to insert themselves. They learn skills for scaffolding student participation, e.g., some use of sentence frames and productive questions (can you say more about that? what do other people think of that idea? what is your evidence? does anyone want to build on that?). They also learn skills to develop good questions for productive discussions and argumentation because not all questions lend themselves to discussion. Additionally, they spend time considering the timing of these discussions, e.g., what do I want my students to get out of this unit? What are the big ideas they're trying to figure out? When will discussion be productive? What will the specific goal of my discussion be? I would love to discuss all of these ideas further.
Your second question (how are you measuring (or planning to measure) gains in students' argumentation skill? Related to part two, what does successful argumentation look like at for elementary students?) is a little easier to answer. We are primarily using an instrument that focuses on teacher and student practices in whole class discussions, called the Science Discourse Instrument (SDI). The SDI was developed by the Stanford portion of our 3-way collaboration for a previous project. We have a publication on it, and we can share that reference below. Coralie and Emily Reigh may add more than what I will include here, but the basic gist is that this instrument measures 3 teacher and 3 student practices. The teacher practices are:
-ASK (asking open-ended questions that have more than one possible response—at least as far as the student audience is concerned—and require discussion)
-PRESS (In productive science discussions, students make their ideas clear and support them with evidence and reasoning. Teachers can support students by pressing them to clarify, elaborate, build on, provide evidence for, or explain the reasoning behind their statements.)
-LINK (In productive science discussions, students link their ideas to previous contributions. Toward this end, teachers can provide opportunities for students to compare, contrast, and critique different ideas that arise. In addition, teachers can highlight interesting or relevant ideas and synthesize different ideas with the goal of bringing the discussion to resolution.)
The student practices are:
-EXPLAIN/CLAIM (Productive science discussions encourage students to move from discussion based on observations to discussion involving ideas, explanations, and claims supported by evidence.)
-CO-CONSTRUCT (Productive science discussions involve students building on each other’s ideas and asking each other questions that clarify, elaborate, or extend what has been said.)
-CRITIQUE (Productive science discussion involves students critically examining the ideas that are shared. Students may either express disagreement or ask questions that challenge one another’s ideas. When students critique one another’s ideas, they should make clear connections between ideas and use evidence and reasoning.)
Our Stanford research team is using this instrument (which includes a rating scale and more nuanced descriptions for each practice) to evaluate changes in teachers' classrooms over time. They video three times in the fall and three times in the spring each year of the project, as well as collect baseline video from teachers before they begin any of the professional learning activities. For students we can only see change within a year because we do not follow the students from classroom to classroom. We are looking at some additional measures as well, such as portions of the embedded assessment system within the school district's adopted science curriculum—items that require argumentation skills.
As far as what "successful" argumentation looks like at the elementary/upper elementary level, this is defined by NGSS. There are specific bullets that are part of the description of the practice of engaging in argument from evidence for the 3-5 grade band. So, ideally you will see these things happening in a classroom. However, we are more concerned with change in student skills than mastery. Are students getting better at it? The goal is really for teachers to meet their students where they are at the beginning of the year and help them improve throughout the year. Simultaneously, this means the teachers have a lot of shifting to do in their practice to support this.
I hope I've helped to address some of your questions.
Best,
Emily
Sally Crissman
Nicole Wong
Sheila Homburger
Many thanks, Emily. I look forward to learning more about the instrument. It's great to see such a focus on helping students understand science as a process, and awesome that they get to participate in that process. I like the idea of students working together to refine their ideas... it's unfortunate that the word argumentation can sound combative : )
Thanks again for sharing this important work. I look forward to seeing how it progresses.
Best, Sheila
Coralie Delhaye
Patrick Moyle
Hi IPT,
I'm excited to see your work unfold and I'm eager to learn more about your work with district leadership. I'm wondering what strategies you used to initially engage with district-level administrators and what work you did to maintain that engagement over time?
Coralie Delhaye
Emily Weiss
PI Improving Practice Together
Hi Patrick,
This is a great question! It's really one of the most fundamental parts of the project if it is to be successful. Initially we collaborated on the proposal so that the proposal represented what the district wanted, along with our other parts of the team's project goals. However, we realized about a year into the project that none of us had ever really worked in a partnership like this before and that all of our expectations for what this should look like were different. Our advisory board helped us with ways to address this, but we were already about a year into the project at that point. One of our biggest issues was that we hadn't figured out clear input opportunities for all project partners to contribute, e.g., who's voice should be at the table? when should it be there? and what are the pieces that each voice should have influence over? We have been working on structures to address these questions. These structures are being developed with the input of all three partners--the district, the professional learning team, and the research and evaluation team.
Our relationship with the district was established long before the proposal development phase. The Lawrence Hall of Science and SCUSD had been working on another science education themed project together for a few years prior to the proposal—one that engaged a vertical leadership team from the superintendent through classroom teachers. Additionally, the Hall and Stanford teams had been collaborating on another project prior to the proposal. These prior relationships allowed for a really positive initial experience when thinking about the current project. Even as some of the leadership transitioned within the district during the project, we were able to rely on our prior relationship to maintain momentum. All, this being said, here are some strategies we have used/are using that are helping the process along, even as we hit many bumps in the road:
1) We have started having quarterly meetings with the full leadership team focused on the district needs/goals and a sustainability plan. This is allowing us to be present and future-focused simultaneously. The district assistant superintendent, elementary curriculum and instruction lead, and science teacher on special assignment are all present for these meetings with the other project principal investigators.
2) We have bi-monthly leadership meetings where at least one member of the district leadership team is present, along with the other PIs, our evaluation team lead, and our project manager/post-doc. These meetings are not usually focused only on the district, but on other partners' needs as well.
3) We have working groups for the development of any project deliverables (for lack of a better word), e.g., agenda development for the Summer Institute and leadership institute and instructional tools/resources. These working teams include members of SCUSD leadership and teachers as appropriate.
Time is definitely a barrier to getting as much input as may be desired (across all of the partners). However, these three steps have helped us maintain considerable engagement from our district partner. I will also try to ask one of our district partners at the administrative level to try to chime into this discussion. I know this is something they are very interested in as well.
Best,
Emily
Coralie Delhaye
Nicole Wong
Thank you for your thoughtful response to Patrick's questions. Something that really resonated with me is when you spoke about the unanswered questions related to the nature of the collaborative effort (e.g., who's voice should be at the table? when should it be there? and what are the pieces that each voice should have influence over?). I see some parallels between the challenges you named and some of the questions that our district-based and school-based professional learning partners have needed to address as they have worked to engage their higher-level administrators and other vital stakeholders in the process of implementing a sustainable PL program that helps their districts transition to NGSS. Thanks for all of your work. This sounds like an exciting project, and I'm excited to learn more about what you find.
Emily Weiss
Coralie Delhaye
Postdoctoral researcher
Hi Nicole,
Thank you for your kind comment, and thank you for sharing about these challenges. It's good to know that we are not alone :-)
I also look forward to know more about the outcomes of your very interesting project. After watching your video, I bookmarked the project's website https://we-mss.weebly.com/ to stay in touch.
Coralie
Joanna Totino
What a great project. I love the focus of argumentation and the collaboration between Higher Ed institutions and the teacher leadership network within the district. Congratulations! Joanna
Coralie Delhaye
Emily Weiss
Emily Weiss
PI Improving Practice Together
Thanks, Joanna!
Acacia McKenna
Director, Science Education Competitions
Thank you for sharing this wonderful project! This academic year, I worked on an education research project involving curriculum development for elementary and middle school students and find this topic very exciting. How do you envision the results from this study will impact future design of a model that can be applied nationally to build school district level capacity?
Emily Weiss
PI Improving Practice Together
Hi Acacia,
Thanks for the question. What you're asking is really at the heart of what we're trying to figure out. Ideally, we'll learn a lot about what it takes to build district capacity to implement high quality professional learning experiences using well-prepared teacher leaders and a district leadership team to support the effort--and to sustain that effort. Many districts have substantial teams of Teachers on Special Assignment with dedicated time to support these sorts of initiatives in ELA and sometimes even math at the elementary level. I think the challenge will come with building out infrastructure for subject areas that don't have this level of support built in, such as science.
Best,
Emily
Catherine McCulloch
Hello Emily and all,
Since you have a long-term partnership to address improvement in these schools while conducting research, I am curious about your next collaboration with all of these partners. What is your process for determining what areas may be the best to tackle together?
Catherine
Coralie Delhaye
Postdoctoral researcher
Hi Catherine,
Thank you for this very interesting question. The processes that we put in place help identify challenges and to tackle them collaboratively at different levels.
Here are two examples:
We also use other processes, some of them are described by Emily in this thread, as a response to Patrick's question about how to engage district administrators.
If you are interested in a particular type of process, I would be happy to share more about what we do and how it allows us to collaborate in identifying, then tackling challenges together.
Coralie
Catherine McCulloch
Thank you, Coralie, If I am understanding your response, this sounds like a great process for identifying challenges related to the CURRENT research and professional development around scientific argumentation. I suppose I am wondering how you will decide what the NEXT topic for research and system improvement will be--after this project is over, and as long as you are expecting to continue to engage with these same partners.
Coralie Delhaye
Postdoctoral researcher
I was indeed referring to how we tackle new challenges that are related to our current topic: building district leadership in scientific argumentation. This is what we will be doing for the next three years: a year for the teacher leaders to implement argumentation, to start feeling confident as leaders and to develop as leaders; a second year for them to co-lead other teachers' professional development with close support from the Lawrence Hall of Science team; and a third year to provide them any support they need to lead professional learning themselves and to ensure the sustainability of the leadership program. So we will not be taking on new topics until our project ends.
This is an excellent question, the idea of continuing our collaboration to tackle new challenges together in the future is really appealing. However, after this project is over we do not have funding (yet?) to continue a partnership. If however we did, I think that we would turn to the very valuable tools provided by other Design Based Implementation Research (DBIR) projects. Here is a toolkit that is particularly interesting to negotiate the focus of joint work in Research Practice Partnerships: http://learndbir.org/tools/negotiating-the-focus-of-joint-work
Emily Weiss
PI Improving Practice Together
Hi Catherine,
We really haven't gotten that far. I think that sort of thing will come organically, e.g., if additional questions pop up during the project or if the district expresses interest in new problems of practice or challenges. It is possible that this may not be the right partnership for some new topics that arise. Because we have a pretty structured format for surfacing ongoing needs and questions through our regular partner meetings, I would hope that potential for ongoing work of the partnership would arise there.
For this project, we are really hoping to construct knowledge and tools that can be used more broadly by the field, so it is possible that our next project would focus more on scalability. But I am not even sure that is the correct place to focus yet.
I think the most honest answer to your question is, "stay tuned."
Best,
Emily
DeLene Hoffner
Lead Teacher
Wonderful sharing and discussions for this video/ project. I'm so impressed with the quality of professional learning represented in this video and overall project. We all can learn from the design and the content of "Improving Practice Together" (IPT)
I wondered if those involved in the project could share some key learnings from your involvement. Any "word to the wise" you could share from things you may do different next time.
Thanks,
DeLene Hoffner
Emily Weiss
PI Improving Practice Together
Hi DeLene,
Yes! I really wish we had spent more time clarifying for/with all partners up front what our various roles were in the project and what we all hoped to gain from participating. This clarity would have empowered all of the teams, especially our district partners, to participate more actively in decision making early on. We have come a long way in this regard, and I think it's drastically improved the potential impact of the project.
Thanks for the great question.
Emily
Coralie Delhaye
Postdoctoral researcher
Thanks for your kind words and your great question DeLene!
I think that something that took us some time was to find the best ways to collaborate in a three-way partnership. We are still figuring out the best ways to do it, and there is room for improvement. However, as I look back in our discussion thanks to the beautiful questions that visitors have asked us about this video, I feel that we achieved to create efficient processes to get input from all partners - the District in particular. We have put a lot of work to make that happen.
Building a common culture and tuning our instruments is a very fulfilling feeling and it is really productive for partnership work. So I think that would be my take-away message: invest early in building a common culture through communication processes that best work to have input from all partners.
It makes me realize how important it is to share about these processes to support the work of research-practice partnerships.
Sally Crissman
Senior Science Educator
I meant to comment earlier on your response to a question about measurement and listed 6 "moves" - 3 each for teacher and students - that you would look for and score. I think you picked some good ones. Improving discussions, either by begin a better facilitator or having students with a better set of skills, is hard (so many factors that can with enhance or deep six a discussion) so if there's a core set to start with and these become ingrained, that's a terrific base on which to build. Of course this brand of argumentation is a key science practice but the discourse skills are useful more broadly.
I look forward to reading about your work as it matures.
Sally
Emily Weiss
Coralie Delhaye
Postdoctoral researcher
Hi Sally,
Thank you so much for this insightful comment. That is also what our teachers tell us. They find that the practices implemented in science class are really valuable to teaching and learning more broadly.
That is the most immediate feedback that we get when they start implementing arguing from evidence in their classrooms.
Coralie
Further posting is closed as the showcase has ended.