Graduate Biennial Program Plan & Assessment Report

Program Information: (Modify table as needed)
Degree/s Assessed PhD in Computer Science
   
College or Administrative Division College of Engineering
Department/School Gianforte School of Computing
Report Submitted By John W. Sheppard
Date Submitted 10/15/19
Assessment Period: Sept 2017 – Aug 2019

Graduate assessment reports are to be submitted biennially. The report deadline is September15th.

Every graduate report must have the following key components.

Part 1: Assessment Plan

Program Learning Outcomes (PLOs): PLOs should be written as specific, measureable statements describing what students will be able to do upon completion of the program. The assessment of PLOs provide feedback on the accumulated knowledge, skills, and attitudes that students develop as they progress through their graduate program. Plans should include PLO’s that would cover all types of graduate programs, depending on the nature of your programs (i.e. Master’s Thesis, Professional, Course work, Doctoral Dissertation, or Certifications). (For help in developing learning outcomes see “Program Assessment Overview”, under Resources on Provost Page: https://www.montana.edu/provost/assessment/program_assessment.html)

Threshold Values: Along with PLOs, plans should include threshold values; minimums against which to assess student achievement for learning outcomes. Threshold values are defined as an established criteria for which outcome achievement is defined as met or not met.

Methods of Assessment & Data Source: Assessment plans require evidence to demonstrate student learning at the program level. This evidence can be in the form of a direct or indirect measure of student learning. Both direct and indirect assessment data must be associated with the program’s learning outcomes. An assessment rubric will also need to be included that demonstrates how evaluation of the data was used to assess student achievement.

Timeframe for Collecting and Analyzing Data: Provide a multi-year assessment schedule that will show when all program learning outcomes will be assessed. As graduate assessment reports are biennial, faculty review of assessment results may only occur every other year, however, annual faculty meeting to review these data and discuss student progress may be beneficial.

Part 2: Program Assessment

The assessment report should identify how assessment was conducted, who received the analyzed assessment data, and how it was used by program faculty for program improvement(s). Assessment reports should also reflect on previous assessment and program improvements by identifying previous program-level changes that have led to outcome improvements.

NOTE: Student names must not be included in data collection. Dialog on successful completions, manner ofassessment (e.g., publications, thesis/dissertation, or qualifying exam) may be presented in table format if they applyto learning outcomes. In programs where numbers are very small and individual identification can be made, focusshould be on programmatic improvements rather than student success. Data should be collected through the year onan annual basis.

Part 1: Program Assessment Plan

A) Program Description (from catalog):

From http://catalog.montana.edu/graduate/engineering/computer-science/#graduatetext

The degree is generally intended for students who have a B.S. or M.S. degree in Computer Science and who want to pursue a research and/or college-level teaching career. The program requires coursework, research, exams and the writing of a dissertation.

Admission to the doctoral program follows the requirements of The Graduate School. Factors that the department uses in its admissions process include GRE scores, TOEFL scores (for non-native English speakers), reference letters, GPA and previous coursework. For more information, please refer to www.cs.montana.edu/future-students-phd.html.

B) Program Learning Outcomes, Assessment Schedule, Methods of Assessment, & Threshold Values

ASSESSMENT PLANNING CHART
PROGRAM LEARNING OUTCOMES Methods of Assessment (Schedule: every 2 years) Threshold Value
Demonstrate technical expertise in an emphasis area. Course grades, qualifying examination, comprehensive examination, dissertation. Students must maintain a 3.0 GPA throughout their degree program and must pass all three examinations to the satisfaction of their graduate committee.
Effectively communicate research results to a scientific audience. Seminars associated with qualifier, comprehensive, and defense; presentations at technical conferences. Students must pass the three examinations. Several courses also include project presentations. Conference presentations demonstrate oral communication. Acceptance of papers in conferences and journals indicate writing communication.
Independently perform quality original research. Publication of research results, publication and defense of dissertation. In addition to the dissertation, each advisor sets their own standard for an expected number of papers submitted and published. A typical number is five.

Part 2: Program Assessment Results

A) What Was Done

  1. Was the completed assessment consistent with the plan provided?

YES__X___ NO_____

If no, please explain why the plan was altered.

  1. Please provide a rubric that demonstrates how your data was evaluated.
Component Expectations not met Meets Expectations Exceeds Expectations
Grade Point Average Student fails to maintain a 3.0 GPA over foundational courses and courses on the program of study Student maintains a 3.0 GPA over foundational courses and courses on the program of study Student maintains a 3.5 GPA over all courses on the program of study
Qualifying examination Student reviews five computer science research papers (both written and orally) but fails to adequately explain the technical problems, the mechanisms behind the technical solution, or the relevant open research questions. Student reviews five computer science research papers (both written and orally) and summarizes paper motivation, the technical problem, the technical solution, and any open research questions. Student reviews five computer science research papers (both written and orally) and clearly summarizes paper motivation, the technical problem, the technical solution, open research questions, the broader impact of the solution in computer science, or the broader society.
Comprehensive examination Student presents a research proposal (written and oral) but fails to motivate the significance of the research, the approach to completing the research, or any preliminary results demonstrating feasibility of the research. Student presents a research proposal (written and oral) and motivates the significance of the research and an approach to completing the research. Students also present preliminary results demonstrating feasibility of the research. Student presents a research proposal (written and oral) and motivates the significance of the research and an approach to completing the research. Students also present preliminary results, together with one or two publications, demonstrating feasibility of the research.
Thesis defense Student fails to motivate the work, explain their technical contribution, demonstrate any novelty in the research, or communicate the results of their research to a technical but non-expert audience. Student motivates their work, explains their technical contribution, and evaluates its performance with data. The solution has some novelty. The student is also able to communicate the results of their research to a technical but non-expert audience. Student motivates their work, explains their technical contribution, and evaluates its performance with data. The solution is novel. The student is also able to communicate the results of their research, clearly, to a technical but non-expert audience, as evidence by insightful questions or comments from the audience.
Thesis Student fails to motivate the work, explain their technical contribution, or demonstrate any novelty in the research. Student motivates their work, explains their technical contribution, and evaluates its performance with data. The solution has some novelty. Student motivates their work, explains their technical contribution, and evaluates its performance with data. The solution is novel.

B) What Was Learned: Results

Please include who received the analyzed assessment data, and how it was used by program faculty for program improvement (s).

  1. Who were the recipients of the analyzed assessment data?

The notification of students passing each of the milestones in the PhD program has been passed to the Graduate School each semester as the milestones are completed. Summary statistics for the past two years are listed here.

  2017-2018 2018-2019
Qualifying examination 4 3
Comprehensive examination 2 1
Doctoral dissertation 1 0
Dissertation defense 1 0

Other statistics collected in this study are listed here by semester.

  Fall 2017 Spring 2018 Fall 2018 Spring 2019
New PhD’s Admitted 8 4 5 2
Average Semester GPA 3.54 3.64 3.61 3.80
Students with Semester GPA < 3.0 0 1 2 0
Students with Semester GPA in [3.0,3.5) 5 2 4 2
Students with Semester GPA >= 3.5 7 11 10 8
  Fall 2017 Spring 2018 Fall 2018 Spring 2019
Average Cumulative GPA 3.61 3.65 3.61 3.64
Students with Cumulative GPA < 3.0 0 0 0 0
Students with Cumulative GPA in [3.0,3.5) 5 5 7 5
Students with Cumulative GPA >= 3.5 15 17 15 15
Total Number of Student Publications 21 21 16 18
Total Number of Student Journal Publications 4 2 5 3
Total Number of Student Conference Publications (refereed) 13 14 8 13
  1. Areas of strength

The assessment measures student performance throughout their time in the program through the GPA and completion of various milestones. In general, course work remains strong as does a consistent practice of student publication. Computer science tends to emphasize refereed conference publications over journal publications due to the rapid pace of change in the field. This emphasis is reflected in the publication statistics above.

  1. Areas that need improvement

The assessment methods and thresholds should be communicated to students through the school’s website. More regular meetings of students with their full committees should be encouraged beyond the one meeting required per year. Students should be encouraged to present their research more often in public forums within the department.

  1. What else was learned?

Time to degree should be examined to find ways to reduce. This is especially relevant for students who, for one reason or another, decided to take a job external to MSU and thus need to manage time between job and degree.

C) Use of Assessment Data

  1. Based on the faculty responses, will there be any curricular or assessment changes (such as plans for measurable improvements, or realignment of learning outcomes)?

YES______ NO__X_____

If yes, when will these changes be implemented?

  1. When will the changes be next assessed?

Not applicable.

  1. What are your goals moving forward?
  • To continue to recruit and grow the PhD student body in support of the growing research activities of current and new faculty.
  • To provide improved financial support through higher stipends and more comprehensive fee/insurance payment to reduce the financial burden on graduate students.

D) Closing the Loop

Reflect on previous assessment and program improvements by identifying previous program level changes that have led to outcome improvements.

  1. What was identified as an area for improvement from the last report?

The last report did not identify any areas for improvement.

  1. What was implemented to improve these outcomes?

Not applicable.

  1. What impact have the changes had (if any) on achieving the desired level of student learning outcomes?

Not applicable.

Submit report to programassessment@montana.edu

This page was converted from https://www.cs.montana.edu/msu-accreditation/2019PhD.pdf