Thank you for the comments and question! At this time, we do not have any structured or pre-assigned content for participants of the coding house. Instead, we found that the Coding House participants (both youth and college students) naturally find computing-related topics to talk about and work on. This might include how to: create content and edit videos for YouTube, put together and fly drones (we have drone kits at the house), create new Scratch content, update their website(s), or creating interactive systems (e.g., an interactive website or simple mobile app).

The most exciting thing we have found is the continuation of the near-peer mentorship between many of the children and their (college student) instructors. We have found that several of the NKC participants and their instructors continue to keep in touch, with the kids often explaining how they are improving/updating their projects from NKC, and sometimes asking programming questions related to these activities. This highlights how engaged our youth are in computing, and the important role that mentorship plays in keeping this engagement active.

Sustaining participation is definitely a challenge! For the NKC you see in the video, we had an incredible 98% attendance rate (and no dropouts).

We attribute this success to the work of our partner school's principal and teacher (who followed up with absentee's parents over the phone) and our "spaghetti night" with students' parents. Before starting NKC, we arranged an evening for parents to come and learn about the upcoming NKC program and why it was critical for their children to learn computing early. During this event, we stressed the importance of attending the program every week. Speaking directly to the parents about being accountable for their children attending the program appears to have helped tremendously with retention.

The challenge now is scaling these efforts to keep these numbers up as we expand the program to multiple schools/locations!

Thank for the response, and congratulations on your attendance rate!! How many programs have you / will you run during the project?

We tried to do the same with our prior research program - inviting parents to come and learn about the program - but on at least three different occasions, no parents joined us. It is difficult to tell which factor was most responsible: the program being about cybersecurity (which is still shrouded in mystery and myth for the general public), or the non-availability of the parents, many who had more than one job.

How did the kids do with knowledge retention week-to-week? Were there opportunities mid-week for them to continue with the activity/concepts?

Thank you for the followup questions!

We are currently running our third iteration, and hoping to run at least once every academic semester (Fall and Spring) while slowly expanding to more schools. We are still working out if we should be at multiple locations (several schools at the same time) or if we should have multiple schools join us at one location (this is what we have done the last two iterations). We have had some students attend two out of the three NKCs, and one student who has attended all three!

Retention does not appear to be too much of an issue week-to-week. We do often briefly summarize what we did the previous week at the beginning of each session. All of our resources (scratch, code.org's html curriculum, and helpgidget.com) are available online for the students to access any time (although we do not explicitly track if they were using these outside of NKC).

Thank you for the positive feedback!

As you mentioned, one of the main goals is to spark and sustain interest in computing for the kids. We have seen evidence for increasing computing knowledge (e.g., applying programming skills to drone programming), but have not yet measure this explicitly. Yes, we are collecting data in several ways to inform next steps. One of these methods is parent interviews. We believe that informed parents lead to better informed children, so we interview (and survey) parents to better understand their knowledge, understanding, and attitudes about computing, so that we can create better resources and talking points to use during the parents' nights.

(I also have more details about our evaluation instruments below, under Feng Liu's question).

Thank you for the question! We use several pre-post tests to evaluate outcomes.

First, for computing attitudes, we will use two paper-based instruments: 1) the Computer Attitudes Questionnaire (CAQ) v5.14 (designed for middle school students), and 2) the Computing Attitudes Survey (CAS) v4.0. Second, for evaluating self-efficacy, we will use the Self-Efficacy with Computer Technologies (SCT) questionnaire as used by Milbrath & Kinzie (2000), and Pamuk & Peker (2008). This measures the participants’ self-reported confidence using different types of computing resources.

We are examining ways to best measure learning outcomes (please let us know if you have any ideas!). Examples include language-agnostic tests such as the FCS1 or SCS1.

We also ask demographic questions on our survey, and include questions relating to career interests, academic interests, computing resources they have at home, and about students' mentors/role-models in computing.

Thanks for sharing these great resources, Michael!

Thank you for the comment Jamie! I can definitely speak about the recruitment procedure. Early in the process, we had an information night with 13+ high school principals from the local public school system. We spoke about the importance of teaching our youth code, and how it was important to teach these kids early. We also spoke about how full (weekly) participation was important, and that parent involvement in making sure their kids attended each week is essential.

After participating schools were chosen (we tried to select evenly across the different areas of our city), the principals at these schools advertise to their teachers, who then advertise to their students. Principals are responsible for collecting applications and permission/consent/assent forms. The school sends outs details about the program, and also advertises a parents' night (described earlier), where parents can come to learn more about the program from us. Parents are invited to observe during NKC days, and almost all students have at least one adult each come to see their final projects on the last ceremony day.

Based on our observations and talks with the kids, there is a mix of doing it for fun, and doing it for their future. I was struck recently, when one of our students, who is currently participating in her third NKC, said: "Before [NKC] I didn't know what I wanted to be [when I grow up], but now, I want to be a software engineer."

As for opportunities outside of NKC, we encourage our kids to continue with more NKCs (as peer mentors or attendees) and to participate in the coding house. In addition, we point them to several online resources where they can learn more about coding (e.g., hour-of-code resources). We also found that some students continue to keep in contact with their near-peer mentors (their college student instructors), updating then on progress they made on their coding projects.

I loved your video and the human-centered iterative approach to designing for authentic issues! Especially the statement by one of your participants, "I'll be able to impact the world somehow by by making tools." Very inspirational! 

Thank you so much for your kind words Nancy! Having a video featuring our students speaking about their experience with the program is a great idea!

Hello Frank! Thank you for the comments and insight into the similarities between our projects. We are not directly measuring the connection between the younger students and older college student instructors. However, we did find that more of the middle school students do report having mentors on our post-assessment compared to their pre-assessment. We do not know for certain if the college instructors are these mentors, but we are fairly that is the case. We plan on having additional space in the next version of our survey for the young students to write-in/elaborate on this point as it is an important measure!

Thank you for sending the link to YPP! It would be great to chat more about our respective programs and results!

This is a great question and something that we are continually trying to learn more about from our parents (and kids). What we really found was effective was to bring in working professionals in the computing/programming field in to talk with the parents (which also works great with our young students). Like you said, many adults have misconceptions/misunderstandings about CS, but learning a little about someone's computing job and being able to ask questions directly seems to mitigate things very quickly. These sessions are also useful for us to learn more about what these misconceptions are, so that we are better prepared for future events.

I would also love to learn more about your research and things you have learned or strategies that you use to approach and inform parents!

Thank you for the questions Daryl! So far, having the main NKC events on Saturdays have been great. We look forward to it, and based on the attendance and overall energy-levels on these days, I am confident saying that the kids are enjoying it too!

All this success can be attributed to the strong ties our university has with our partners. The Urban League is excellent in exciting the local community and generating positive buzz towards the program(s). The Newark Public Schools (NPS) and leadership is amazing with the investment in their students, providing much needed resources including equipment, personnel, facilities, and transportation. I think this is the key to make these types of programs self-sustaining: a group of deeply invested stakeholders who really want this to succeed and keep it succeeding. Once you have these core stakeholders, we found that others naturally get excited and contact us to want to contribute (e.g., we have had dozens of refurbished laptops donated to us by the National Cristina Foundation and Alliance for Technology Refurbishing & Reuse for use in our program and Coding House). In a similar fashion, we have many computing-related professionals contact us asking how they can help/contribute. Sometimes, the best way is for them to come in and talk to our parents/kids as potential role-models!

I watched your video and digital forensics is great! We're always looking for additional ways to keep our students engaged (especially for those returning students), so we should definitely chat more about your work!