{% hint style="warning" %} This section is especially a work in progress, these numbers and targets are still being adapted. {% endhint %}
There are six program success tiers that can be applied to quantify a course:
- Platinum - This is the best possible course experience, world-class and exceptional. NPS is 80+ and completion rates are over 90%.
- Gold - This is an exceptional course, with an incredible student experience, and high completion rates. NPS is 70+ and completion rates are over 80%.
- Silver - This is a great experience, with an overwhelmingly positive student experience. NPS is 50+ and completion rates are over 70%.
- Bronze - This is a good experience, with a majority positive student experience. NPS is 40+ and completion rates are over 60%.
- Needs Improvement - This is a mixed experience, with a high variance of experiences. NPS is 20+ and completion rates are over 50%.
- Poor - This is a course with an often negative or neutral experience for many students, with a high variance of often mediocre experiences. NPS is under 20 and completion rates are under 50%.
For every course we run, how do we practically evaluate success and improve our courses over time? We are looking at the following top-level metrics at first:
- Cohort Completion - What % of students overall complete the course objectives successfully? What % from admitted, what % from week 3? How do URMs compare to total students on these metrics?
- Subjective Satisfaction (NPS) - How do students self-report their likelihood to recommend the course to others (i.e "How likely are you to recommend this course to other students?")
- Perceived Impact - How do students self-report the impact of the course in specific ways we want the course to help. (i.e "How well did this course increase your confidence in technical interviewing?")
- Student Proficiency - How well does the program achieve it's desired proficiency outcomes? For example, if this is an inspire course, how well are students inspired to continue? If this is a software fundamentals course, how well are students reporting on interview success? If this is a special topics course, how are students faring in their ability to build real projects during and after the course?
- Follow-through - How likely are students that complete the course to take additional courses and complete their journey over multiple semesters?
These four distinct metrics help us understand the various ways our programs are working, and where we need to improve. We should be asking ourselves always:
- How do we ensure that a higher percentage of students that are admitted into a course (and make it to week 3), are likely to complete the course objectives successfully?
- How do we ensure that a higher percentage of students are promoters and believers, and are likely to evangelize our courses to others?
- How do we ensure that a higher percentage of students report the course had a higher impact on their confidence around key course objectives?
- How do we ensure that a students are achieving higher levels of progress in their personal proficiency at the end of the course?
What makes for an amazing student experience?
- Proper leveling - Setting up students for success in the program by making sure students that enter the class are appropriately prepared.
- Pacing and workload - Topics and concepts are introduced at a reasonable pace, and provide a reasonable workload
- Story and narrative - Provide a clear and compelling story for students, with proper context and motivational elements.
- Progressive spaced repetition - Concepts are carefully introduced and applied, repeated over time, and delivered with proper context.
- Student Support - Fast and accurate technical and career support such that students feel well supported when they are confused or things go wrong.
- Meaningful - The topics, projects and skills encountered in this course are significant to the student and relevant to their specific goals.