Academic Program Assessment Report: PhD in Computer Science, 2024–2026
Biennial: 2024-2025/2025-2026
Academic Year(s) Assessed: 2024-2025, 2025-2026
College: NACOE
Department: Gianforte School of Computing
Department Head: John Paxton
Submitted by: Binhai Zhu
PROGRAM(S) ASSESSED
| Majors | Minors, Options, etc. |
|---|---|
| PhD in Computer Science | |
Section 1. Past Assessment Summary.
Summarize findings from last time this PLO(s) was assessed. Include findings from the last assessment of this PLO that influenced this cycle’s assessment approach. (Note: this is NOT a summary of what was done during last year’s assessment cycle unless you are assessing the same PLOs again this year; this is a summary of the last time you assessed the PLO(s) that you are re-assessing this cycle. You will connect the summary to the reflection portion in Section 7: Closing the Loop. Table has been provided to support the intention of this question.)
| Which PLO(s) are you assessing this cycle? Please include the description. | When was the last time you assessed this PLO(s)? | Action items from last assessment of this PLO(s). | Were past action items addressed in this reporting cycle? This will be discussed in depth in Section 4. |
|---|---|---|---|
| #3 - Independently perform quality original research | 2024 | Count PhD publications in AI (and machine learning) separately | Yes |
Section 2. Actionable Research Question for Your Assessment.
Are students performing quality original research (indicated by refereed publications)? How about in the areas of AI and machine learning?
Section 3. Assessment Plan, Schedule, and Data Sources.
- Did you change the previously established Assessment Plan Schedule. If yes, how was it changed?
Yes. We modified how we are assessing PLO #3 (Independently perform quality original research) to specifically count how many of the total number of publications were AI related due to the increasing importance of AI. This is the first time that we count AI publications specifically.
- Please provide a multi-year assessment schedule that will show when all program learning outcomes will be assessed, and by what criteria (data). List all of your PLOs in full for reference - not just the ones being assessed in this cycle. Add rows as necessary.
ASSESSMENT PLANNING SCHEDULE CHART
| PLO# | PROGRAM LEARNING OUTCOME | 2026-2027 | 2027-2028 | 2029-2030 | 2030-2031 | Data Source |
|---|---|---|---|---|---|---|
| 1 | Demonstrate technical expertise in an emphasis area. | X | X | X | X | Course grades, Pass rate on qualifying exam, comprehensive exam and dissertation defense. |
| 2 | Effectively communicate research results to a scientific audience | X | X | X | X | Presenting papers at technical conferences. |
| 3 | Independently perform quality original research | X | X | X | X | Publication of research results, publication and defense of dissertation. |
- What are the threshold values for which your program demonstrates student achievement? Provide a rationale for your threshold values.
| PROGRAM LEARNING OUTCOME | Threshold Value | Data Source |
|---|---|---|
| (1) Demonstrate technical expertise in an emphasis area. | 90% of students maintain a 3.0 GPA throughout their degree program. 75% of students pass the subsequent exams when they take them: qualifying exam, comprehensive exam, dissertation defense. | Course grades, Pass rate on qualifying exam, comprehensive exam and dissertation defense. |
| (2) Effectively communicate research results to a scientific audience. | 50% of students will author or co-author a refereed conference or journal paper that is accepted during the 2-year assessment period. | Accepted papers at technical conferences or journals. |
| (3) Independently perform quality original research. | 90% of students will have at least three conference or journal publications at the time they graduate. 50% of students will have at least 5 publications at the time they graduate. | Accepted papers at technical conferences or journals. |
Section 4 What Was Done?
-
Was the completed assessment consistent with the program’s assessment plan? If not, please explain the adjustments that were made.
Self-reporting Metric (required answer):
Yes
- How was the data collected and analyzed and by whom? Please include method of collection and sample size.
- Our administrative associate collected GPA data for all finishing PhD students during the assessed period.
- Dr. Zhu, our PhD coordinator, collected information for all PhD students who took the qualifying exam, the comprehensive exam or the dissertation defense.
- Dr. Zhu collected publication data for all PhD students during the assessed period from GSoC faculty.
- Dr. Zhu analyzed the data.
- The number of PhD students ranged from 25-30.
- Please provide a rubric for each PLO being assessed that demonstrates how your data was evaluated. (See Instructions document for an example of what this could look like.)
| Indicators | Expectations not met | Meets Expectations | Exceeds Expectations |
|---|---|---|---|
| Grade Point Average | Student fails to maintain a 3.0 GPA over foundational courses and courses on the program of study. | Student maintains a 3.0 GPA over foundational courses and courses on the program of study | Student maintains a 3.5 GPA over all courses on the program of study |
| Qualifying exam | Student reviews five computer science research papers (both written and orally) but fails to adequately explain the technical problems, the mechanisms behind the technical solution, or the relevant open research questions. | Student reviews five computer science research papers (both written and orally) and summarizes paper motivation, the technical problem, the technical solution, and any open research questions. | Student reviews five computer science research papers (both written and orally) and clearly summarizes paper motivation, the technical problem, the technical solution, open research questions, the broader impact of the solution in computer science, or the broader society. |
| Comprehensive exam | Student presents a research proposal (written and oral) and fails to motivate the significance of the research, the approach to completing the research, or any preliminary results demonstrating feasibility of the research. | Student presents a research proposal (written and oral) and motivates the significance of the research and an approach to completing the research. Students also present preliminary results demonstrating feasibility of the research. | Student presents a research proposal (written and oral) and motivates the significance of the research and an approach to completing the research. Students also present preliminary results, together with one or two publications, demonstrating feasibility of the research. |
| Dissertation defense | Student fails to motivate the work, explain their technical contribution, demonstrate any novelty in the research, or communicate the results of their research to a technical but non-expert audience. | Student motivates their work, explains their technical contribution, and evaluates its performance with data. The solution has some novelty. The student is also able to communicate the results of their research to a technical but nonexpert audience. | Student motivates their work, explains their technical contribution, and evaluates its performance with data. The solution is novel. The student is also able to communicate the results of their research, clearly, to a technical but nonexpert audience, as evidence by insightful questions or comments from the audience. |
| Dissertation | Student fails to motivate the work, explain their technical contribution, or demonstrate any novelty in the research. | Student motivates their work, explains their technical contribution, and evaluates its performance with data. The solution has some novelty | Student motivates their work, explains their technical contribution, and evaluates its performance with data. The solution is novel. |
Section 5. What Was Learned.
The following is the summary of statistics for the past two years.
| 2024-2025 | 2025-2026 | Comments | |
|---|---|---|---|
| Qualifying Exams | 5 | 6 | One student left the PhD program after earning an MS |
| Comprehensive Exams | 1 | 3 | |
| Doctoral Dissertation | 3 | 2 | |
| Doctoral Defense | 3 | 2 | |
| All PhD Student Publications | 24 | 29 | |
| PhD Student AI Publications | 14 | 14 | This is new and is a subset of All PhD Student Publications |
We also collected the GPA’s of the 5 students who successfully defended their dissertations: 3.50, 3.58, 3.69, 3.90 and 3.97. All of them presented at least once at international conferences besides their public presentations of the qualifying exam, comprehensive exam and dissertation defense.
- Based on the analysis of the data, and compared to the threshold values established, what was learned from the assessment? (Relate this to your research question in Section 3.)
- GPA assessment and milestone completion are accurate metrics of student abilities across a range of computer science areas.
- The number of PhD students in 2024-2025 and 2025-2026 are 26 and 30 respectively, higher than in the previous assessment cycle (i.e., 19 and 20). In part, this is due to some new faculty hires.
- The number of students finishing the comprehensive exam and dissertation defense stayed roughly the same, but the number of students taking the qualifying exam (11, all well passing the minimum 3.0 GPA bar) was much higher than in the previous assessment cycle (i.e., 2). This implies that more new PhD students are starting to step into their dissertation research.
- PhD publications in this report (24,29) are lower than in the previous report (41,37), possibly due to the fact that in the previous assessment cycle there were 7 students finishing their dissertation defense (compared to 5 here) and our newer PhD students are just starting their dissertation research.
- The number of PhD publications in AI are counted separately for the first time. We plan to use this as an indicator for our research in AI in the subsequent assessment (including the upcoming 7-year cycle Program Review).
- What areas of strength in the program were identified from this assessment process?
- PhD productivity in machine and machine learning remains strong. Publication-wise, the rate of publications over the corresponding totals are 58.5% and 48.3% respectively for 2024-2025 and 2025-2026. (Note that we count this separately for the first time.)
- PhD productivity in cybersecurity and software quality is stable.
- What areas were identified that either need improvement or could be improved in a different way from this assessment process?
- PhD publications are lower than in the previous report, possibly due to many new PhD students starting. Many of these newer students have just passed the qualifying exam and are starting their dissertation research.
- PhD productivity in algorithms is lower than in the previous assessment cycle, with no one taking the comprehensive exam and dissertation defense. The reason is similar as several PhD students have just started in this area.
- We did not hire new faculty in systems or networks, hence the shortage of research expertise in these areas remains to be rectified.
Section 6. How We Responded.
- Describe how “What Was Learned” was communicated to the department, or program faculty. How did faculty discussions re-imagine new ways program assessment might contribute to program growth/improvement/innovation beyond the bare minimum of achieving program learning objectives through assessment activities conducted at the course level?
- This document was circulated in the week of October 11, 2026. The feedback was then incorporated into the report.
- How are the results of this assessment informing changes to enhance student learning in the program?
- As part of continuing discussions on improving the PhD program, results of this assessment as well as feedback from student and faculty experiences inform discussions during school meetings and retreats.
- We will revisit requirements for qualifying exams and comprehensive exams with the goal of better assessing student progress and readiness for the next milestone.
- If information outside of this assessment is informing programmatic changes, please describe that.
- The school’s industry advisory board provides high-level feedback annually.
- It might be helpful to form a PhD advisory committee that includes external members. The committee could provide different aspects from the industry advisory board.
- What support and resources (e.g., workshops, training, etc.) might you need to make these adjustments?
- If we form a PhD advisory committee, some extra resources to invite an external person to campus for a workshop should be identified.
Section 7. Closing the Loop(s).
Closing the Loop refers to continuous improvement and planning for future action. Reflect on the program learning outcomes and how they were assessed in the previous cycle (refer to #1 of the report). What was learned in this cycle and what actions will the program take in planning for future assessment of the PLO(s).
- Self-Reporting Metric (required answer): Based on the findings and/or faculty input, will there be any changes made (such as plans for measurable improvements, realignment of learning outcomes, curricular changes, etc.) in preparation for upcoming assessments?
NO ꟷ However, we could collect slightly different information in the coming two years and then possibly adjust our indicators in section 4. Specifically, it might be helpful to identify our most significant publication categories (e.g. AI, algorithms/theory, cyber, etc.) and count each one separately.
- In reviewing the last report that assessed the PLO(s) in this assessment cycle, what changes proposed were implemented and will be measured in future assessment reports? What action will be taken to improve student learning objectives going forward?
- Students have been informed about publication expectations.
- Students are required to meet with their committee once a year, as a school-wide practice.
- Students are encouraged to present their research more often in public forums. This has been implemented in many of the labs (e.g., Applied Algorithms, CompTag, Cybersecurity and NISL), but not in school-wide seminars.
- Have you seen a change in student learning based on other program adjustments made in the past? Please describe the adjustments made and subsequent changes in student learning.
- More students are prepared to achieve key degree milestones (i.e., qualifying exam).
- If the program sees anything emerging from this assessment cycle that it anticipates would be a factor or an item of discussion in its 7-year program review cycle, please use this space to document that for future reference.
Not applicable.
OTHER COMMENTS
Use this area to documented anything not covered in previous sections that department wants to remember for future reference.
