Program Assessment Report: MS in Computer Science, 2022–2024
Academic Year(s) Assessed: 2022-2023 and 2023-2024
College: Engineering
Department: School of Computing
Department Head: John Paxton
Submitted by: Sean Yaw
Program(s) Assessed
List all majors (including each option), minors, and certificates that are included in this assessment – addor subtract rows as needed – please use official titles:
| Majors | Minors, Options, etc. |
|---|---|
| MS in Computer Science | Thesis Track, Courses-Only Track |
1. Past Assessment Summary.
The previous assessment report for the School of Computing’s MS program (2020-2021) found that student GPA makes an adequate assessment metric in determining student abilities across a range of computer science areas. The previous assessment also identified the need for an improved metric to assess the following learning outcome: “Effectively communicate knowledge to a scientific audience.” This finding influenced us to use grades from the presentation component of a required MS course (CSCI 532) to assess that learning outcome in the current assessment.
2. Action Research Question.
Do students have technical expertise in computer science fundamentals?
3. Assessment Plan, Schedule, and Data Sources.
- Please provide a multi-year assessment schedule that will show when all program learning outcomes will be assessed, and by what criteria (data).
| PROGRAM LEARNING OUTCOME | 2024-2025 | 2025-2026 | 2026-2027 | 2027-2028 | Data Source* |
|---|---|---|---|---|---|
| 1 - Demonstrate technical expertise in the fundamental areas of computer science. (Thesis and courses-only tracks) | X | X | X | X | Course Grades |
| 2 - Integrate their knowledge of disparate computer science subjects. (Thesis and courses-only tracks) | X | X | X | X | Course Grades |
| 3 - Effectively communicate knowledge to a scientific audience. (Thesis and courses-only tracks) | X | X | X | X | Presentations and written report assignments in computer science courses |
| 4 - Communicate research effectively to a scientific audience. (Thesis track) | X | X | X | X | Thesis defense |
| 5 - Perform original research. (Thesis track) | X | X | X | X | Thesis |
- What are the threshold values for which your program demonstrates student achievement?
| PROGRAM LEARNING OUTCOME | Threshold Value | Data Source |
|---|---|---|
| 1 - Demonstrate technical expertise in the fundamental areas of computer science. (Thesis and courses-only tracks) | Students must have a 3.0 GPA on the courses listed on the Program of Study. | Course grades |
| 2 - Integrate their knowledge of disparate computer science subjects. (Thesis and courses-only tracks) | Students must have a 3.0 GPA on the courses listed on the Program of Study. | Course grades |
| 3 - Effectively communicate knowledge to a scientific audience. (Thesis and courses-only tracks) | 80% of students pass courses in-class presentation or written report assignments. | Grades on presentations and written report assignments in computer science courses |
| 4 - Communicate research effectively to a scientific audience. (Thesis track) | 80% of thesis students pass their defense | Thesis defense |
| 5 - Perform original research. (Thesis track) | 80% of thesis students have their thesis accepted by their graduate committee | Thesis |
4. What Was Done.
- Self-reporting Metric (required answer): Was the completed assessment consistent with the program’s assessment plan? If not, please explain the adjustments that were made.
☒ Yes
☐ No
- How were data collected and analyzed and by whom? Please include method of collection and sample size.
- Data was collected by Ana Gaskill (School of Computing Administrative Associate) and Sean Yaw (Interim MS Program Coordinator), and analysis was conducted by Sean Yaw.
- GPA and grade data was collected from DegreeWorks for all active students in the assessment period.
- Grades on presentations and written report assignments in the course (CSCI 532) required to be taken by all MS students were collected from the course instructors during the assessment period (Spring 2023 – Brendan Mumey, Fall 2023 – Sean Yaw).
- There were 28 MS students in the program in 2022-2023 and 30 MS students in the program in 2023-2024.
- Analysis was done by binning students according to the rubric and calculating percentages.
- Please provide a rubric that demonstrates how your data were evaluated. (Delete example below and replace with program’s assessment-specific rubric.)
| Indicators | Beginning - 1 | Developing- 2 | Competent- 3 | Accomplished- 4 |
|---|---|---|---|---|
| 1 - Demonstrate technical expertise in the fundamental areas of computer science. (Thesis and courses-only tracks) | GPA between 3.0 and 3.2 | GPA between 3.2 and 3.4 | GPA between 3.4 and 3.7 | GPA between 3.7 and 4.0 |
| 2 - Integrate their knowledge of disparate computer science subjects. (Thesis and courses-only tracks) | GPA between 3.0 and 3.2 | GPA between 3.2 and 3.4 | GPA between 3.4 and 3.7 | GPA between 3.7 and 4.0 |
| 3 - Effectively communicate knowledge to a scientific audience. (Thesis and courses-only tracks) | Applicable assignment grades between 70% and 80% | Applicable assignment grades between 80% and 90% | Applicable assignment grades between 90% and 95% | Applicable assignment grades between 95% and 100% |
| 4 - Communicate research effectively to a scientific audience. (Thesis track) | Successfully Defends thesis | |||
| 5 - Perform original research. (Thesis track) | Successfully Defends thesis |
5. What Was Learned.
- Based on the analysis of the data, and compared to the threshold values established, what was learned from the assessment?
Presented below are the assessment values applied to the rubric for each Indicator.
Indicator 1 - Demonstrate technical expertise in the fundamental
areas of computer science.
Indicator 2 - Integrate their knowledge of disparate computer science
subjects. (Thesis and courses-only tracks for both)
NOTE: 3 students did not have GPAs in each period due to pass/fail
thesis credits, so the sample size of students is 25 for AY 22-23 and 27
for AY 23-24.
| Deficient GPA below 3.0 |
Beginning – 1 GPA between 3.0 and 3.2 |
Developing- 2 GPA between 3.2 and 3.4 |
Competent- 3 GPA between 3.4 and 3.7 |
Accomplished- 4 GPA between 3.7 and 4.0 |
|
|---|---|---|---|---|---|
| Number (percentage) of students in range (FY22-23) | 2 (8%) | 2 (8%) | 0 (0%) | 5 (20%) | 16 (64%) |
| Number (percentage) of students in range (FY23-24) | 2 (7%) | 2 (7%) | 1 (4%) | 4 (15%) | 18 (67%) |
Indicator 3 - Effectively communicate knowledge to a scientific
audience. (Thesis and courses-only tracks)
NOTE: 5 applicable students took CSCI 532 in FY22-23 and 10 in
FY23-24.
| 0 Applicable assignment grades below 70% | Beginning – 1 Applicable assignment grades between 70% and 80% |
Developing- 2 Applicable assignment grades between 80% and 90% |
Competent- 3 Applicable assignment grades between 90% and 95% |
Accomplished- 4 Applicable assignment grades between 95% and 100% |
|
|---|---|---|---|---|---|
| Number (percentage) of students in range (FY22-23) | 0 (0%) | 0 (0%) | 2 (40%) | 2 (40%) | 1 (20%) |
| Number (percentage) of students in range (FY23-24) | 0 (0%) | 0 (0%) | 4 (40%) | 4 (40%) | 2 (20%) |
Indicator 4 - Communicate research effectively to a scientific
audience.
Indicator 5 - Perform original research. (Thesis track for both)
NOTE: 3 students attempted thesis defenses in AY22-23 and 2 in
AY23-24.
| 0 Unsuccessfully Defends thesis | Beginning – 1 |
Developing-2 | Competent-3 | Accomplished- 4 Successfully Defends thesis |
|
|---|---|---|---|---|---|
| Number (percentage) of students in range (FY22-23) | 0 (0%) | - | - | - | 3 (100%) |
| Number (percentage) of students in range (FY23-24) | 0 (0%) | - | - | - | 2 (100%) |
Assessment takeaways:
- Students in the program are successful at demonstrating technical expertise in fundamental CS areas and in integrating their knowledge with 84% (FY22-23) and 82% classified as competent or accomplished.
- Students demonstrated acceptable performance in communicating to a scientific audience with 60% being classified as competent or accomplished and the remaining 40% being classified as developing.
- Students had a 100% success rate in thesis defense during the evaluation period.
- What areas of strength in the program were identified from this assessment process?
- Demonstrating technical expertise in fundamental CS areas and integrating their knowledge were identified as strengths, given the GPA performance demonstrated.
- Research performance for thesis track students was also a strength, with effective thesis defenses.
- What areas were identified that either need improvement or could be improved in a different way from this assessment process?
- No areas were identified as being below target thresholds.
- Expanding the assessment of PLO 3 to include presentation grades from the other course required of MS students (CSCI 538) would work to expand the assessment data and would cover all required courses.
6. How We Responded.
- Describe how “What Was Learned” was communicated to the department, or program faculty. How did faculty discussions re-imagine new ways program assessment might contribute to program growth/improvement/innovation beyond the bare minimum of achieving program learning objectives through assessment activities conducted at the course level?
This assessment was shared with GSoC faculty and then discussed at a faculty meeting on October 10, 2024. Recommendations concerning the evaluation of PLO 3 have been incorporated.
- How are the results of this assessment informing changes to enhance student learning in the program?
Particular attention will be paid to tracking CSCI 532 and CSCI 538 presentation grades to assist in the assessment of PLO 3.
- If information outside of this assessment is informing programmatic change, please describe that.
Our external industry advisory board sometimes meets with graduate students during their annual meeting and when they do, they provide us with feedback.
- What support and resources (e.g. workshops, training, etc.) might you need to make these adjustments?
N/A
7. Closing the Loop(s).
Reflect on the program learning outcomes, how they were assessed in the previous cycle (refer to #1 of the report), and what was learned in this cycle. What action will be taken to improve student learning objectives going forward?
- Self-Reporting Metric (required answer): Based on the findings and/or faculty input, will there any curricular or assessment changes (such as plans for measurable improvements, or realignment of learning outcomes)?
☐ Yes
☒ No
- In reviewing the last report that assessed the PLO(s) in this assessment cycle, what changes proposed were implemented and will be measured in future assessment reports?
The last assessment identified the need to better assess PLO 3. To this end, in this assessment, we used presentation grades from CSCI 532, one of the two courses required of all MS students, to assess the outcome of PLO 3.
- Have you seen a change in student learning based on other program adjustments made in the past? Please describe the adjustments made and subsequent changes in student learning.
No (not applicable).
