Annual Program Assessment Report: MS in Computer Science, 2020–2021
Academic Year Assessed: 2020-2021
College: Engineering
Department: Computer Science
Submitted by: Mike Wittie
Undergraduate Assessment reports are to be submitted annually by program/s. The report deadline is October 15th.
Graduate Assessment reports are to be submitted annually by program/s. The report deadline is October 15th.
Program(s) Assessed:
List all majors (including each option), minors, and certificates that are included in this assessment:
- MS in Computer Science
☑ Have you reviewed the most recent Annual Program Assessment Reportsubmitted and Assessment and Outcomes Committee feedback? (please contactAssistant Provost Martha Peters if you need a copy of either one).
The Assessment Report should contain the following elements, which are outlined in this template:
- Assessment Plan, Schedule, and Sources
- What was done this assessment cycle – including rubrics, how data was collected, and who analyzed it
- What was learned – including areas of strength and areas for improvement
- How we responded
- Closing the loop
Sample reports and guidance can be found at: https://www.montana.edu/provost/assessment/program_assessment.html
1. Assessment Plan, Schedule and Data Source.
- Please provide a multi-year assessment schedule that will show when all programlearning outcomes will be assessed, and by what criteria (data). (You may use thetable provided, or you may delete and use a different format).
| PROGRAM LEARNING OUTCOME | 2021-2022 | 2022-2023 | 2023-2024 | 2024-2025 | Data Source* |
|---|---|---|---|---|---|
| Thesis and courses-only track: Demonstrate technical expertise in the fundamental areas of computer science. | ✓ | ✓ | ✓ | ✓ | Course grades |
| Thesis and courses-only track: Integrate their knowledge of disparate computer science subjects. | ✓ | ✓ | ✓ | ✓ | Course grades |
| Thesis and courses-only track: Effectively communicate knowledge to a scientific audience. | ✓ | ✓ | ✓ | ✓ | Presentations and written report assignments in computer science courses. |
| Thesis track: Communicate research effectively to a scientific audience. | ✓ | ✓ | ✓ | ✓ | Thesis defense |
| Thesis track: Perform original research. | ✓ | ✓ | ✓ | ✓ | Thesis |
*Data sources can be items such as randomly selected student essays or projects, specificallydesigned exam questions, student presentations or performances, or a final paper. Do not usecourse evaluations or surveys as primary sources for data collection.
- What are the threshold values for which you demonstrate student achievement?(Example provided in the table should be deleted before submission)
| PROGRAM LEARNING OUTCOME | Threshold Value | Data Source |
|---|---|---|
| 1. Thesis and courses-only track: Demonstrate technical expertise in the fundamental areas of computer science. | Students must have a 3.0 GPA on the courses listed on the Program of Study. | Course grades |
| 2. Thesis and courses-only track: Integrate their knowledge of disparate computer science subjects. | Students must have a 3.0 GPA on the courses listed on the Program of Study. | Course grades |
| 3. Thesis and courses-only track: Effectively communicate knowledge to a scientific audience. | 80% of students pass courses in-class presentation or written report assignments. | Presentations and written report assignments in computer science courses. |
| 4. Thesis track: Communicate research effectively to a scientific audience. | 80% of thesis students pass their defense | Thesis defense |
| 5. Thesis track: Perform original research. | 80% of thesis students have their thesis accepted by their graduate committee | Thesis |
2. What Was Done
-
Was the completed assessment consistent with the program’s assessment plan?
☑ Yes ☐ No
-
If no, please explain.
-
How were data collected and analyzed? (Please include method of collection andsample size).
- GPA as reported in Degree Works has been used to approve students for graduation. Students need a 3.0 to graduate, which lets us know that our graduating students meet the learning outcomes 1 and 2. The graduation rate from our MS program in the review period was 4.7% with 88.7% continuing to year two and only one student on academic probation going into Spring’22.
- Presentations and/or written report assignments are assigned in all our graduate level courses. They are a significant portion of the grade, so indirectly, students who pass these courses do well on these assignments. The pass rate in 500-level classes in the review period was 92%.
- All our thesis-track students must submit a thesis manuscript and defend it. Our successful thesis defense rate is 100%.
- Please provide a rubric that demonstrates how your data was evaluated. (Example provided below should be deleted before submission – your rubric may be verydifferent, it just needs to explain the criteria used for evaluating student achievement).
| Indicators | Beginning - 1 | Developing- 2 | Competent- 3 | Accomplished- 4 |
|---|---|---|---|---|
| 1. Thesis and courses-only track: Demonstrate technical expertise in the fundamental areas of computer science. | GPA between 3.0 and 3.2 | GPA between 3.2 and 3.4 | GPA between 3.4 and 3.7 | GPA between 3.7 and 4.0 |
| 2. Thesis and courses-only track: Integrate their knowledge of disparate computer science subjects. | GPA between 3.0 and 3.2 | GPA between 3.2 and 3.4 | GPA between 3.4 and 3.7 | GPA between 3.7 and 4.0 |
| 3. Thesis and courses-only track: Effectively communicate knowledge to a scientific audience. | Completes most written assignment tasks/ Presentation covers most basic points | Completes all written assignment tasks/ Presentation covers all basic points | Completes all written assignment tasks/ Presentation covers all basic points Provides some synthesis | Completes all written assignment tasks/ Presentation covers all basic points Provides substantial synthesis |
| 4. Thesis track: Communicate research effectively to a scientific audience. | Thesis accepted for publication | |||
| 5. Thesis track: Perform original research. | Defends thesis |
This type of rubric can be used for all levels of assessment (the anticipated evaluation score may varyaccording to the course level). Some rubrics/assessments may be more tailored for courses (e.g.designed to assess outcomes in upper division courses or for lower division) and therefore the scoresmight be similar across course levels. Or, if you are assessing more basic learning outcomes, you mightexpect outcomes to be established earlier in the academic career.
NOTE: Student names must not be included in data collection. Dialog on successful completions, mannerof assessment (publications, thesis/dissertation, or qualifying exam) may be presented in table format ifthey apply to learning outcomes. In programs where numbers are very small and individual identificationcan be made, focus should be on programmatic improvements rather than student success. Data shouldbe collected through the year on an annual basis.
3. What Was Learned
- Based on the analysis of the data, and compared to the threshold values established,what was learned from the assessment?
- GPA assessment is an accurate metric for student abilities across a range of computer science areas.
-
What areas of strength in the program were identified?
- Many student projects are informed by real-world problems relevant to the industry. This link between student projects and industry topics is appropriate and desirable for an MS program.
- We have had a growing number of students enter the accelerated MS program introduced since the last assessment cycle. Currently there are 4 CS undergraduates pursuing the accelerated MS degree.
-
What areas were identified that need improvement?
- We did not adequately assess learning outcome 3 in relevant courses.
4. How we responded
- Describe how “What Was Learned” was communicated to the department, or programfaculty.
- We will share this report with the faculty at our annual retreat in August 2022.
- How are the results of this assessment informing changes to enhance studentlearning in the program?
- We will communicate with the faculty, during the annual retreat, how to perform assessment of learning outcome 3 in their courses.
- If information outside of this assessment is informing programmatic change, pleasedescribe that.
- We consider feedback on our MS program from the industry advisory board. The board recommended more focus on systems, including cloud and blockchain technologies. In response to this we have proposed a new course, CSCI 521: Distributed Computing, to help us cover these topics.
5. Closing the Loop
- In reviewing last year’s report, what changes proposed were implemented and will bemeasured in future assessment reports?
- The last report mentioned that we were considering an MS in Cybersecurity. In this report period, we have been hiring faculty and developing courses to establish that degree program.
- Have you seen a change in student learning based on other program adjustmentsmade in the past? Please describe the adjustments made and subsequent changes instudent learning.
- We have eliminated the comprehensive exam for the MS courses-only students and evaluate their performance based on the GPA alone. This procedure significantly simplifies graduation.
- We have streamlined the committee formation in that courses-only students may have only one faculty advisor on their committee.
Submit report to programassessment@montana.edu
