Hello Hong, thank you for your comment & question. I think POGIL helps students to develop skills they need in team projects, which they often don't learn in traditional courses. My software engineering course uses both POGIL and team projects. My intro courses uses POGIL, and I encourage students to use pair programming for homework, but there are not team projects.

To evaluate individual learning, many POGIL instructors have a short individual quiz after an activity, or embed clicker-style questions in the activity. Other POGIL instructors (including me) use more traditional exams and homework assignments to measure learning. Within the POGIL community, the ELIPSS project is focused on ways to measure process skills (communication, teamwork, critical thinking).

I am also involved with the NSF OpenPath project to involve students in humanitarian FOSS projects, which often involves team project work. In our workshops, we encourage faculty to focus evaluate on the process (did students do the right things) rather than product (did they get code working, and was it accepted by the project), since the latter may be belong the students' control.

Does this help? Do you have other questions?

Sorry, my reply appears as a separate comment below.

Hi Danielle, thanks for your comment & questions.

To recruit faculty participants, we do 1/2 and 1 day workshops. We started with workshops at conference (e.g. at SIGCSE, the main US conference for CS education). We tried smaller conferences, but it was hard to get enough people for an effective workshop. We also realize that many faculty aren't able to attend conferences, particularly if they are at community colleges or comprehensive institutions without deep financial resources. Thus, we now offer "workshops on request" - if local organizer(s) can register 16+ teachers of whom 8+ teach intro CS, we will provide materials, a facilitator, & their travel expenses for a workshop. This has been a win-win - the local organizers handle publicity & logistics through their local institution & connections, and we provide a workshop experience. We plan to contact dept head / dean level people at specific institutions and see how that works too.

For preparation time, part of the challenge is developing or adapting materials. We have several sets of materials for CS1, and continue working to make them easier to adapt by splitting them into pieces that can easily be rearranged, indicating learning objectives & prerequisites, etc. We're also trying to foster teams of faculty to develop materials for other courses, as has happened for a variety of courses in chemistry, biology, and health sciences. Another part of preparation time is planning how to use a specific activity in a class. We try to help faculty think about this during a summer workshop, and provide an experienced mentor for them to talk to as they start using POGIL. Preparation also gets much easier with practice - I've even talked to some faculty who feel that POGIL requires less pre-class prep than a lecture.

Does this make sense? Do you have more questions?

Thank you for the follow up. I really like the idea of the workshops on request as this can connect faculty at community colleges, MSI, and other institutions that may not have the resources. Are you analyzing how/what impact these "mini POGIL communities" have on faculty adopting/implementing POGIL after the workshop?

Another challenge is that CCs, MSIs, tend to have heavier teaching loads and more part-time faculty, both of which make more challenging to adopt POGIL & develop activities.  We're also trying to find better ways to contact the right people at such institutions to tell them about POGIL and workshops.

The POGIL Project does keep a list of people who have attended workshops and every year or so sends out an implementation survey to find out what people are doing. But some people ignore such surveys, so it's hard to know what people do. I think of POGIL as a collection of practices that work really well together, and I suspect that many workshop participants adopt or adapt a subset of practices, so their teaching has changed in a meaningful way but they might not self-report as adopting POGIL.

For IntroCS POGIL, we are interviewing our participating faculty, and we've talked about trying to interview CS faculty who attended a workshop but didn't participate in our project, to better understand why they did or didn't adopt POGIL (or some practices). That might help us understand more about how faculty decide to try and/or persist, and what interventions might help them.

I've seen POGIL used in several ways in intro programming courses, such as:

• Provide a short code example that uses a new syntax element (if-else, loop, array, etc) and ask questions that prompt teams to understand how it works and then make specific changes or use the syntax in a new context. For example, show a while loop and have them figure out when and how the test controls behavior.
• Use a code or non-code example to show a problem that motivates new syntax, and once students understand the objective, introduce the new syntax that does it. For example, I wrote an activity on unit testing that gradually leads to the JUnit framework.
• use an interesting problem, and then ask questions that prompt teams to analyze it and develop requirements for a program to solve it (perhaps as a later homework assignment). For example, I have an activity where students gradually develop simulations for infectious diseases.

The faculty participants in IntroCS POGIL are mostly using sets of activities by Helen Hu and Chris Mayfield. To see some examples, check out activities at http://cspogil.org/Computer+Science+1

There are certainly connections & overlap between POGIL and other approaches to teaching and learning. POGIL combines a set of practices (teams, role, inquiry, learning cycles, teacher facilitation, etc) that seem to work really well together but also work well alone or in other combinations. If there's something magic about POGIL, I think it's the structure - when I started with POGIL, I felt like it helped me achieve my teaching goals more quickly than I was doing on my own by trial and error.

Hi Marcelo, thanks for your questions.

Some people start to write new activities after a 1-day workshop, but we encourage prospective authors to attend a longer workshop. The POGIL Project (http://pogil.org) offers 3-day workshops that include sessions on writing - the structure of activities, learning objectives, models, author coaching, etc. The POGIL Project also runs a Writers Retreat, and reviews activities (using a set of rubrics) to determine how well they align with POGIL principles.

Within the CS POGIL community, we encourage people to share activities and submit them for review. The activities listed at http://cspogil.org include activities at various levels of development - drafts to activities that use some elements of POGIL to activities that have been reviewed & endorsed by The POGIL Project.

Clif,

Thanks for the additional details. It sounds like you have a great system in place. The writers retreat is a great idea. Can you say more about the rubrics that are used? Are there also some post-activity follow-up questions to help refine activities?

One rubric focuses on the structure of the activity. Are the learning objectives clear? Are students likely to achieve them? Are the models (examples) appropriately complex? Do the questions guide students to explore them model, invent their own understanding, and apply it? Is the sequencing logical? Is the wording appropriate to the audience?

Another rubric focuses more on the process skills. Does the activity engage all students and prompt students to work together? Does it prompt students to use specific skills (e.g., critical thinking, problem solving) and to reflect on what they have learned?

So the review of an activity is like the review of a journal paper - it considers whether the content is appropriate, and provides feedback and suggestions to improve it.

When I do POGIL in class, I always have a printed copy for reference, and at the end of class it's often covered with notes and ideas for revision, based on what I see and hear during class. Here's a question that confused some teams. Here's a place where teams got stuck, and a question or observation that helped them move forward. Here's a question that could be skipped or elaborated depending on timing. It's amazing what we can learn from our students if we stop talking, and watch and listen. :-)