In accordance with the Royal Military College of Canada (RMC) Institutional Quality Assurance Process (IQAP), this Final Assessment Report (FAR) provides a synthesis of the external evaluation and internal response and assessments of the undergraduate programme in Computer Science offered by the Faculty of Science. This report identifies the significant strengths of the programs, together with opportunities for improvement and enhancement, and sets out and prioritizes the recommendations that have been selected for implementation.
This report includes an Implementation Plan that identifies who will be responsible for approving the recommendations set out in the Final Assessment Report; who will be responsible for providing any resources entailed by those recommendations; any changes in organization, policy or governance that will be necessary to meet the recommendations and who will be responsible for acting on those recommendations; and timelines for acting on and monitoring the implementation of those recommendations.
Overview of Programme Review Process:
The Program Self-Study Reports were completed in August 2019. For the program under review - the BSc in Computer Science - it contained the degree level expectations, an analytical assessment, course outlines, program-related data, survey data from the Office of Quality Assurance and appendices with sample examinations and CVs of faculty members of the program. Two arm’s-length external reviewers (Dr. Roger Villemaire, Département d'informatique, UQAM, and Dr. Will Traves, Department of Mathematics, United States Naval Academy) were selected from a list of possible reviewers and approved by the Deans of Science. An internal reviewer, Dr. Ryley Beddoe, Department of Civil Engineering, was also selected for participation on the ERC. They reviewed the self-study documentation and conducted a site visit to RMC on 12 and 13 December, 2019. During the site visit, the ERC met with the Vice-Principal, Academic, Dr. Phil Bates, the Associate Vice-Principal, Research, Dr. Mike Hennessy, the Dean of Science, Dr. Gregg Wade, the Department Chair, Maj David Eisenhauer, Computer Science program coordinator, Dr. François Rivest, the Head Librarian, Ms. Sarah Toomey, as well as several members of department faculty. The ERC also had the opportunity to meet with several students currently enrolled in the program. The ERC subsequently produced a Report based on the Self-Study and site visit. The report was circulated to department members and discussed at a departmental meeting.
The reviewers submitted their report in March 2020. In their report, they found that the overall quality of the programs offered is comparable to programs at other Canadian universities.
Significant Strengths and Areas of Concern of the Program:
The ERC identified a number of strengths of the BSc in Computer Science Program:
- Committee members indicated they were extremely impressed by the dedication of all the faculty and staff (including military and civilian) to the unique mission of the RMC;
- The department offers a comprehensive undergraduate degree that compares favourably with those of much bigger institutions;
- The committee was impressed by the efforts to cooperate across departments and within programmes to provide an excellent educational product; and
- Students have the opportunity to take engaging and rigorous elective/optional courses in current subjects.
The ERC identified a number of areas of concern for the BSc in Computer Science Program, and suggestion for program enhancements:
- A number of faculty members expressed concerns about the current level of teaching and service loads relative to the staffing level. They also reported that hiring staff to teach courses was extremely laborious and time-consuming, in part because the process and requirements for hiring are not clear and change frequently;
- The largest current and future challenge is the delivery of this program with a reduced full-time faculty of two. While they are certainly managing the delivery of the program at the moment, this is not sustainable; and
- Another important issue which surprised the Committee is that there is a very low number of graduates, as reported in the Cyclical Review Report, who use their computing skills/degree in their military careers.
The Program Chair, after consultation with faculty and staff in the programs, submitted a response to the Reviewers’ Report in June 2022. The Dean of Science prepared this Final Assessment Report in August, 2022. Specific recommendations are discussed, and follow-up actions and timelines provided.
Summary of the Reviewers’ Recommendations with Dean’s Responses:
The ERC identified a number of areas of concern or issues that require attention. These issues are discussed in the order that they appear in the ERC Report:
Recommendation 1
The Department needs to work with senior leadership at the RMC to develop a long-term HR plan to address the department’s anticipated needs and to smooth the hiring process.
Departmental Response:
The need to hire more regular faculty is by far the most pressing problem in the department. Anything that RMC can do to help smooth the hiring process (reducing the amount of paperwork done by the department to better focus on candidates and interviews) is more than welcome.
Dean’s Response: At the time of this review there were only two Computer Science faculty in the department. The recent hires of two CS faculty have now doubled the CS complement and significantly reduced sustainability risk.
Recent experience suggests that indeterminate hiring of UTs, from the perspective of the Dean’s office (which is intimately involved in all stages of the process), runs relatively smoothly. Permission to replace retiring faculty is provided in a timely fashion, the process for competitive hiring is clear and straightforward, and the development of Letters of Offer following candidate selection is efficient. It is however acknowledged that the staffing process within the Department of National Defence is more onerous than one might find at a civilian university. There is nothing that the institution can do to change the official processes.
On the general subject of workload, RMC is currently supporting a Joint Committee reviewing workload distribution across all Faculties of the College. The findings of the committee should inform and address concerns related to teaching and service loads.
Recommendation 2
The process of engaging sessionals needs to be streamlined in order to reduce the administrative burden on the Department. One way of doing this, as well as addressing the service task burden, would be to hire more terms as opposed to casuals or part-time exclusions.
Departmental Response:
Anything that can be done to reduce the onerous tasks of recruiting, hiring and mentoring of the department’s unusually large number of term professors and sessional instructors is welcome.
Dean’s Response: The Dean agrees that hiring of Term UTs/Sessional instructors can be fraught with frustrations, including very long lead times (incompatible with identification of suitable individuals to fill positions), burdensome paperwork, etc. Recent changes to the responsibilities of the Faculty of Science CR-04 Claim Clerk, adding assistance with departmental staffing administration and an increase in the number of people working in Faculty Services, should help to alleviate these pressures.
Recommendation 3
Sufficient numbers of term or indeterminate professors should be engaged to ensure the delivery of first year courses by more experienced faculty.
Departmental Response:
Fully agreed; see 1.1. In general, we have a significant number of non-permanent faculty (sessionals, terms or military faculty) delivering 1st-year courses. It is very hard to increase the number of experienced faculty delivering first-year courses, given that near half of our courses overall are delivered by non-permanent UTs. As an example, for the 2021-2022 academic year, 7 members of the department were responsible for delivering MAE/F101. Of these, 2 were inexperienced (in that they have been teaching at the College only recently). Also, only 3 were permanent UTs. The department will make every effort to keep the number of experienced faculty delivering first-year courses relatively high, given the available resources.
Dean’s Response: The Dean agrees that experienced indeterminate faculty are required to effectively deliver MAE/F101 and other introductory courses. At minimum, the course coordinator would be such an individual. While it is likely unrealistic that the large service teaching requirements of the department will ever be fully satisfied by indeterminate faculty, identifying means by which to build a cadre of experienced Term UTs would be of great value. This could be accomplished through utilization of longer-term Term UT contracts, guaranteeing that those in a tenuous employment position are engaged over multiple years.
Recommendation 4
The department should develop a deliberate approach to recruiting potential students into the two UG degree programs. Both the CS and Mathematics programs should hold a ‘course fair’, where students could come and read course outlines and talk with professors prior to pre-registration and selection of majors.”
Departmental Response:
This is a good suggestion which would however require further time and effort from the faculty. Each year, in January, the Department meets the first-year students in science for program briefings. When in-person briefings are possible, we should encourage science students to come to the department and discuss with the professors (which would be a department tour in other departments, where there are actual things to be shown). Ideally, first-year engineering students should also be part of the science program briefings, but it would require the college approval.
Dean’s Response: The Dean concurs that recruitment of undergraduate and graduate students into both the Mathematics and Computer Science programs should be a high priority. Creative recruitment tactics such as those proposed are certainly encouraged.
Recommendation 5
The department will need to hire between two and three full-time faculty in computer science to maintain their program.
Departmental Response:
This is certainly true. As of June 2022, the department has hired two new CS professors. We will also need to hire in mathematics. We need to hire in the field of analysis, and possibly in analytics (this is tied to 3.5 below). It is important that MCS be able to hire earlier, so that new professors can start before or exactly when the faculty member leaves. This is not what has happened lately (cases in point are our previous two hiring cycles). Approval to hire can be requested/granted as soon as the formal notice to retire is submitted. However, we have been experiencing significant delays in the hiring process itself.
Dean’s Response: The Dean concurs that additional hiring to buttress the CS program is a priority, and worked with the Dept Head to hire two new CS faculty in 2022.
As discussed above, the Dean’s experience with recent indeterminate UT hiring processes is that the process itself is reasonably well streamlined. Naturally, competition within certain fields of research may introduce difficulties in attracting appropriate individuals and may delay a process. While negotiation of starting salary and other benefits is possible to some degree within the Public Service framework, it is clear that RMC does not have the leverage in terms of hiring benefits that other universities may have.
Assigning military faculty with a Computer Science background to the department would be of great benefit.
Recommendation 6
Arrange a public presentation of final student assignments, for instance in image processing, artificial intelligence, and simulation, to showcase what can be achieve in the program.
Departmental Response:
Agree. 4th year projects and graduate research is a great way to showcase what is possible. It is also good for the military students to present in front of peers. Having a central place to reference the past projects would help the department showcase the work of the students.
Dean’s Response: The Dean concurs.
Recommendation 7
Develop and promote different ‘specialization paths’, including pure CS, systems (in connection with EE) and theory (in connection with the Math program). Make sure students get the term by term schedule of the required courses.
Departmental Response:
This is in line with our current plan and we hope we could do that. However, with our current number of professors in CS, this is not possible at this time. Furthermore, this would require the department be allowed course planning two or three years in advance, as specialization paths are currently done through optional courses (and approving an optional course is dependent on the number of registrations in it).
Dean’s Response: The Dean supports this recommendation, with the proviso that overspecialization resulting in a large number of specialized courses and concomitant requirements for additional teaching resources should be avoided. The Dean also reminds the department that the minimum course enrollment numbers must also be respected.
Implementation Plan
| Recommendation | Proposed Follow-up | Responsibility for Leading Follow-up | Timeline for Addressing Recommendation |
|---|---|---|---|
|
1. The department needs to work with senior leadership at the RMC to develop a long-term HR plan to address the department’s anticipated needs and to smooth the hiring process. |
A 5-year Departmental Business Plan is being developed and discussed with leadership. | Dept Head | Dec 2022 |
| 2. The process of engaging sessionals needs to be streamlined in order to reduce the administrative burden on the Department. | Faculty claims clerk will be trained to assist with sessional and TA hiring administration. | Dean of Science | May 2023 |
| 3. The department will need to hire between two and three full-time faculty in computer science to maintain their program. | Two new indeterminate CS faculty were recently engaged. Two recent retirements from the department could be filled with an additional CS faculty members should the department so decide. | Dept Head | May 2023 |
| 4. The department should explore new approaches to recruiting students | Develop marketing/outreach/ recruitment plan. | Dept Head | May 2023 |
| 5. The department should hire 2-3 new full-time Computer Science faculty for sustainability | Recruit and hire additional CS UTs (achieved), develop a strategy for training/recruitment of CS Milfacs. | Dean/Dept Head | May 2023 |
| 6. Arrange a public presentation of final student assignments, for instance in image processing, artificial intelligence, and simulation, to showcase what can be achieve in the major | Arrange a public presentation of final student assignments. | Dept Head | May 2023 |
| 7. Develop and promote different ‘specialization paths’. | The Department has initiated an internal discussion of potential specialization paths. | Dept Head | May 2024 |
Conclusion:
The ERC Report provided positive feedback on the outcomes of the undergraduate program in computer science. It confirmed that the Department is delivering an academically rigorous program to its students and that its standards meet those of similar programmes in Ontario. However, the ERC did identify areas that had room for improvement, and RMC is, or has already taken, taking steps to address the issues raised.
The Dean of Science, in consultation with the Department Chair, is responsible for monitoring the Implementation Plan.