Final Assessment Report and Implementation Plan
Per the Royal Military College of Canada (RMC) Institutional Quality Assurance Process (IQAP), this Final Assessment Report (FAR) provides a synthesis of the external evaluation and internal response and assessments of the graduate program offered by the Faculty of Social Sciences and Humanities in Master of Public Administration (MPA). This report identifies the significant strengths of the programs, together with opportunities for improvement and enhancement, and sets out and prioritizes the recommendations selected for implementation.
This report includes an Implementation Plan that identifies who will be responsible for approving the recommendations set out in the Final Assessment Report; who will be responsible for providing any resources entailed by those recommendations; any changes in organization, policy or governance that will be necessary to satisfy the recommendations and who will be responsible for acting on those recommendations; and timelines for working on and monitoring the implementation of those recommendations.
Overview of Program Review Process
The Program Self-Study Report was completed in June 2021. For the program under review, Master of Public Administration, the report contained the degree level expectations for this program, an analytical assessment of the program, course outline, program-related data, survey data from the Office of Quality Assurance and appendices with sample examinations and CVs of faculty members. Two arm's-length external reviewers (Dr Malcolm G. Bird, Associate Professor, Department of Political Science, University of Winnipeg and Dr Nelson Michaud, Professeur titulaire, Ecole nationale d’administration publique) were selected from a list of possible reviewers and approved by the Dean of Graduate Studies. Dr Frank Zeman, Associate Professor in the Department of Chemistry and Chemical Engineering at RMC, was the internal reviewer for the ERC. They reviewed the self-study documentation and conducted an on-site visit to RMC from 7 to 8 March 2023. During the site visit, the ERC met with the Vice Principal-Academic (VPA), the Director of Quality Assurance, the Dean of Graduate Studies (DGS), the MPA Program Chair and Co-Chair, various faculty members, students registered in the program, the librarian-in-chief, and the acting VP Research. The ERC also toured the RMC facilities, including the relevant administrative units, the library and other physical infrastructure.
The reviewers submitted their report on 15 April 2023. In their report, the ERC finds that the MPA is a strong program that meets the requirements of a public administration program with an appropriate accent on the reality of the Canadian Armed Forces (CAF).
Significant Strengths and Areas of Concern of the Program
The ERC identified several strengths of the MPA program:
- The most valuable feature of this program is its remote course delivery model since it offers flexibility to the student body.
- The offering of elective course options allows students to tailor their courses to their specific interests and strengths and is a tremendous asset to this program. The wide breadth of electives is due to the ability of students to take electives from other departments at the RMC as well as from Queen's University; and
- The plan to offer three specific streams to complement the general MPA Program is a very significant innovation to this program. Offering three clear streams provides a competitive advantage compared to other MPA degrees.
The ERC identified some areas of concern for the MPA program:
- Online support for students is limited, and they need consistent access to electronic books and journal articles. Electronic access is a considerable problem given the remote character of the program;
- The faculty complementing its current "Home" in the Department of Political Science and Economics does not sufficiently support this program. While some faculty members are supportive and active in the program, there appears to be a general, departmental reluctance to support this program fully and to ensure its long-term viability, including making changes to its operations and implementing recommendations for program alterations from previous reviews;
- The lack of administrative support services and associated faculty has significantly affected the MPA program's academic quality. It has meant that those involved with the program spend time keeping it running rather than improving content and delivery.
The Program Chair, after consultation with faculty and staff in the programs, submitted a response to the ERC Report on 30 May 2023. The Dean of Graduate Studies prepared this Final Assessment Report on 6 June 2023. Specific recommendations are discussed, and follow-up actions and their timelines are provided.
Summary of the Reviewers’ Recommendations with Dean’s Responses
The ERC identified several areas of concern or issues that require attention. The problems are discussed in the order in which they appear in the ERC Report:
Recommendation 1
Improve governance structure. The MPA would enormously benefit from a dedicated "home" department focusing on its unique public administration and graduate student orientation. It does not need to be a whole department per se but could be a sub-unit in the Division of Graduate Studies (DGS). Such an arrangement would allow for faculty and administrative staff to focus on the specific needs of this fully remote, virtually-offered graduate program. Despite its remote teaching character, the program requires a physical and organizational 'anchor' so that it can effectively articulate its needs to senior administration. This reorganization would allow individuals to focus more fully on the program's educational and organizational requirements. We suggest an office cluster and a dedicated meeting room.
Departmental Response
The MPA programme greatly suffers from relying on the Departments of Political Sciences and Economics and Management to offer faculty members to teach the course. There is a constant battle between the three master's programmes offered in SSH (MPA, MBA and WS) as it is typically the same professors assigned to provide graduate-level courses. Further, like most at RMC, these departments suffer from budget cuts in the hiring process that went from a mainstream SWE budget model to an O&M budget model. When assigning their faculty to the MPA, they typically have to compensate for the lack of available professors in their respective Departments for hiring. It is a substantial administrative burden and financial strain. When they do not fulfil their minimal obligations to assign faculty to the MPA due to sickness, parental leave or sabbatical, or simply due to the lack of faculty in their Department, the MPA has to hire sessional instructors, which is a high risk in terms of continuity and content quality. It is highly recommended to create a Department dedicated to the MPA. It would require hiring 12 Faculty members specialized in the three main streams of the programme.
Dean’s Response
The Dean understands the programme's issues with providing sufficient faculty for the MPA programme. The fact that the MPA programme is interdisciplinary and does not have a dedicated "home" in the Faculty of Social Sciences and Humanities exacerbates these problems.
As the reviewers mention in their report, the Division of Graduate Studies could be an option as a "home" for the programme. The current MPA programme representative resides in the Division of Graduate Studies.
Improved collaboration between the Deans of SSH and Graduate Studies could minimize the administrative burden of contracting and assigning professors for the programme. Still, it would require additional resources to be given to Graduate Studies.
The suggestion to create an additional department with dedicated faculty may need to be revised. This option would significantly increase the resource implication. However, reliance on contract hires to provide the majority of the programme is also not sustainable and risks the programme's quality. To this end, the Deans must work together to ensure that the appropriate levels of support and faculty are assured.
VPA’s response
Although the VPA understands the essence of this ERC recommendation, the Chair's suggestion of creating a new department dedicated to the MPA program is impossible. Instead, improved collaboration between the involved Deans, Departments Heads and the program Chair would allow for better course delivery and scheduling planning.
Recommendation 2
Dedicated and specialized MPA faculty. The program requires faculty members dedicated to the discipline of public administration in general and to this program's well-being in a specific sense. Whether through mid-term (for instance, three years) cross-appointments or secondments of faculty that have some of their teaching assignments (potentially, three courses per year) dedicated to the program. Moreover, it needs a Chair and a Co-Chair who can devote their energies to its operations and students. The Chair requires more decision-making authority in allocating teaching and administrative resources. This authority will bring more consistency to the teaching complement and reduce uncertainty and administrative burdens associated with having to bargain yearly with other departments for faculty members' teaching assignments. It would also allow for teaching relief associated with developing or revamping courses.
Departmental Response
The Chair has absolutely no say in the faculty assignments and needs to manage, on an annual basis, poor quality of teaching, availability and supervisory capacity. It tarnishes the quality and reputation of the programme among DND and other federal employees who are the typical students registered in the programme. The solution to this problem is highlighted in the response to Recommendation 1.
Dean’s Response:
This recommendation is similar to Recommendation 1. The Chair and Co-Chair must have the decision-making authority in allocating teaching. The Dean's response to Recommendation 1 applies here, where the Deans of SSH and Grad Studies must collaborate to ensure quality professors are involved in the teaching. I suggest the creation of a "roster" of vetted faculty from which the Chair/Co-Chair can select to teach in the programme. The Chair, in consultation with the Department Chairs, could identify and choose the appropriate professors that will teach in the MPA programme before faculty teaching loads are finalized and approved by the VPA.
Recommendation 3
Dedicated and sufficient administrative and technical support: This program is complex regarding its administrative and technical needs. The elimination of the Division of Continuing Studies (DCS) and the course and Informational Technology (IT) support it provided has yet to be adequately replaced via other means, resulting in much of these requirements being placed on the shoulders of the Program Chair. The lack of support is not sustainable. The program has a large student body, complicated teaching arrangements, and challenging contracting processes required to hire sessional teachers. It is a fully remote program with a diverse student body and has considerable technical requirements. A program of this size and complexity requires between 2.5 and 3.0 full-time employees. We suggest an administrator (AS 2) oversee the program in addition to the existing Program Support Coordinator (CR 3) to manage the student relationships as well as a dedicated IT person.
Departmental Response
With the new requirements in security clearances and the complexity of the student body, being in the vast majority (more than 90%) part-time students, the programme needs a student representative for everyday functions but also a dedicated administrative assistant who could manage security clearances, relationship with the registrar office and budget. Further, the programme is entirely Moodle-based, and no IT employee at RMC is assigned to support students and professors at the graduate level. All technical support falls under the responsibility of a sole individual for the entire faculty, undergraduate and graduate students. A dedicated Technical Officer for the Graduate studies would be essential for the better functioning of the programme. The programme chair now provides tutorials to the faculty and contractual instructors.
Dean’s Response:
The Dean agrees that a dedicated IT employee is required to support the students and faculty. IT support is needed across the entire college and is familiar to the MPA. The Chair should not be required to provide tutorials to faculty or contractual instructors to use the learning management platform.
The lack of IT support is a college-wide issue. Resolving it would improve the MPA programme and the other, mainly online interdisciplinary graduate programmes.
The Dean disagrees with adding an AS-02 to oversee the programme (see recommendation 7). The Division of Graduate Studies currently has an AS-02 that oversees all of the graduate programmes including the MPA. Adding another AS-02 would duplicate the work.
VPA’s response
CIS offers Moodle support, and the RMC Writing Centre has an educational designer and a dedicated IT tech to help with RMC Moodle courses. The MPA program should take advantage of these available resources.
Recommendation 4
Implement the three thematic streams for the MPA: We ardently support the introduction of the three (3) specific streams within the MPA Program, focusing on Defense Administration, Emerging Technology (Cyber-related issues), and Health in addition to the General MPA stream. These streams align well with both emerging vital areas in public administration (Technology and Health) in a general sense. Still, these two areas and Defense Administration are also directly relevant to RMC's role in supporting CAF and Canada's defence and security sector, more broadly. The General MPA Stream will provide the flexibility (including the thesis and research project) to allow students to tailor their program to meet their interests. RMC has the faculty capacity to support this new model adequately.
Departmental Response
The streams are already theoretically in place. The next move will be to make them available on the main MPA webpage and registration system. With the approval of the Graduate Studies Committee, the degree will be able to mention the specialized stream in which the students have completed their degrees. There will also be a possibility of having a general degree (no stream).
Dean’s Response
The Graduate Studies Committee members are currently canvassing their respective departments to add the discipline/specialization to the student's parchments. Everything is in place to action this recommendation in short order. The three streams are in place and need some advertising.
Recommendation 5
Implement certificate programs: Building on the new stream-based model, we suggest that the MPA program adopt certificate programs centred on the new streams model. Students would be required to take Canadian Government and Public Policy (MPA 521) and Organizational Theory (MPA 569), forming a base knowledge of Canada's governing structures and management principles. They would then augment this with three electives drawn from their stream of choice. The certificate would provide a valuable credential to students who need help to dedicate themselves to the entire MPA program while augmenting their professional skill acquisition. At the same time, the certificate would be a gateway into the formal MPA program for students who may, at a later date, wish to upgrade their credentials. Given the student complement at RMC, many of whom work full-time jobs and have family obligations, such a credential would have considerable appeal.
Departmental Response
This recommendation will be dealt with during the next academic year.
Dean’s Response
The Dean agrees with the response. Adding certificate programmes will increase the programme's attractiveness and help with accessibility. Adding certificate programmes must be stressed more.
Recommendation 6
Integrate the Joint Command and Staff Program (JCSP) into the MPA: We also recommend creating a pathway for JCSP students to join the MPA program and that they receive elective course credits towards their MPA. This pathway is similar to the current NSP, whereby students who have completed it are eligible for admission into the MPA and receive course credits for their work. It is our understanding that JCSP students were eligible for four elective credits in the MPA program. It would expand the MPA student pool and provide JCSP students with access to other credentials and a degree with wider credential recognition. Such efforts align well with RMC's and the MPA's role in supporting the educational aspirations of CAF members.
Departmental Response
The MPA chair and associate chair went through a rigorous process to allow the NSP students to take four courses in the MPA to obtain a degree. It was an extremely complex process due to the need for more availability of teaching material at CFC. The same procedure will be used for the JCSP programme very soon. Timelines are dependent on the availability of CFC'S teaching material.
Dean’s Response
The Dean encourages collaboration between the programmes offered at CFC and the graduate programmes offered at RMC. It is essential to provide pathways for all eligible students to obtain graduate degrees in Defence Studies and the MPA.
Recommendation 7
Improve tracking/monitoring of student progress: This program is complex, given its remote teaching character and many students attend part-time. Student progress must be monitored and tracked to ensure that students are making their way through the program promptly and to identify students who may require additional assistance. The attrition rate is not problematic, but this status must be taken seriously in the future. Hence, close monitoring is essential for reducing student attrition. Dedicated and sufficient administrative support is critical to do this, as this is outside the scope of the duties of the Chair and Co-Chair positions. This recommendation can be considered a reiteration of serial 16 from the 2016 SSR.
Departmental Response
A dedicated CR-3 and AS-2 to the programme will alleviate this issue, per the Departmental response to Recommendation 3.
Dean’s Response
The CR-3 effectively tracks and monitors student progress and reports directly to the Chair/Co-Chair. The Dean disagrees with adding an AS-2 to track or monitor students.
Recommendation 8
Improve the post-course evaluation feedback mechanisms: Timely, honest, and effective feedback from students is essential for maintaining the academic rigour and real-world applicability of the course content. We have seen the results of general surveys regarding the program but did not receive an evaluation for a given course. If these do not exist yet, we recommend that the program implement an online, anonymous course evaluation system so that students can provide an honest assessment of their course relatively soon after its completion.
Departmental Response
This recommendation is outside the realms of the MPA Chair and belongs to the Quality Insurance programme. To better understand students' needs, the MPA chair plans to hold student meetings to present the programme and address challenges and recommendations at the start of every semester.
Dean’s Response
The Dean will discuss the options available with the Quality Assurance office to offer "class climate" surveys to all graduate students so that they can provide timely, honest and effective feedback.
Recommendation 9
Improved library support: The remote character of this program does not diminish the need for strong support from RMC's Library, and as it currently stands, the library is unable to provide optimal support to the MPA's students. There needs to be a formal Distance Learning Librarian and limited resources allocated to RMC's Library regarding human resources and online content provision. The lack of resources means the library cannot provide students with ongoing support through a dedicated portal or individual or adequate library-source material. We recognize the resource limitations of the library and note that the Chief Librarian is aware of these challenges. We also suggest that increasing the support for remote learners be integrated better within any strategic restructuring of the RMC library and its learning support resources.
Departmental Response
A straightforward process to provide free and automatic access to Queen’s Stauffer Library would alleviate this issue, given the systemic stagnation of RMC’s library.
Dean’s Response
A Memorandum of Understanding between RMC and Queen's currently exists for providing shared services. The MOU was renewed in 2020 and is subject to renewal soon.
The Dean will suggest that the Principal adds access to Queen’s Library to the list of shared services for the graduate programs.
Chief Librarian’s Response
Establishing a Distance Learning Librarian position would go a long way to providing better support to distance students. On the other hand, the library has few indications that faculty teaching distance courses or distance students contact the RMC Library for any reason.
Better communication would be an excellent first step. As soon as the new Chair of the MPA program has been announced in July, the Chief Librarian will contact them to discuss Library related issues and plans for improvement.
Past MOUs with Queen's established some access to Queen's library resources, mainly print. Expanding access may be discussed as part of renewing the MOU between RMC and Queen's. Still, it is doubtful that Queen's has the legal right to extend access to electronic resources of interest to the MPA program.* This is due to the publishers' licenses that govern Queen's subscriptions. Many licenses prohibit an institution from providing access to those not associated with that institution. Some licenses allow walk-in use on the library premises. Enabling remote access to non-authorized users is almost unheard of, which accounts for most people associated with the MPA distance program. Legal and licensing questions aside, any agreement would not be cost-neutral.
*A review of Queen's list of databases on public administration shows the RMC library provides access to five of eleven titles directly or indirectly through the Canadian Armed Forces Virtual Library (CAFVL).
Recommendation 10
Promote programs better within CAF, DND and other related security organizations and the federal government. With a bit of expenditure of resources, this program could do more to promote itself amongst its target pool of candidates within the CAF, and Canada's related security establishments, more generally. We suggest targeted advertising, strategic attendance at student/learning fairs and having a set of dedicated faculty members to answer questions and to promote the program better physically. We also note that this program could help fill a considerable need within Canada's MPA learning infrastructure for remote-based and more flexible learning models but recognize that this might be outside of its function in supporting CAF members in their knowledge acquisition efforts.
Departmental Response
Before the pandemic, the MPA student rep attended one fair a year. It was the goal of the MPA programme Chair to attend more meetings at the Parliament and engage stakeholders. Hopefully, this will be possible soon, and a budget line will be assigned to this function. The MPA took advantage of the pandemic to offer different lectures by subject matter experts to engage its students and these experts in the MPA. This line of effort will also be continued and assigned a budget.
Dean’s Response
The Dean aims to promote all graduate programmes at RMC via advertisement and attending student/learning fairs. The newly assigned Associate Dean of Graduate Studies is an expert in marketing. They will be instrumental in increasing our visibility within the CAF and the Public Service.
Recommendation 11
Develop specific requirements for MPA 505 and MPA 507. In an earlier review, Serial 9 added specific requirement criteria for course exemption to be adopted for MPA 505 and MPA 507 listed on the MPA portion of the RMC website. Clarifying the requirements would assist students in determining their eligibility and provide for a consistent application of standards across students and Chairs.
Departmental Response
Students receive a spreadsheet describing the requirements and get support from the MPA Chair and Associate Chair when filling them out. No changes are required at this level. The matrix will be presented during the tri-annual meetings with the students, which should help them understand the requirements if needed.
Dean’s Response
The Dean endorses this response. No further action is required.
Implementation Plan
| Recommendation |
Proposed Follow-up |
Responsibility for Leading Follow-up | Timeline for Addressing Recommendation |
|---|---|---|---|
| 1. Improve governance structure |
|
|
|
| 2. Dedicated and specialized MPA faculty | Same as above | Same as above | Same as above |
| 3. Dedicated and sufficient administrative and technical support |
MPA Chair and Associate Chair will seek assistance from CIS for IT support. Faculty teaching in the course will avail themselves to the available shared resources at the college such as the Educational Designers in the Writing Center and the technical support for the Learning Management System (Moodle) |
|
Summer 2024 |
| 4. Implement the three thematic streams for the MPA | Streams to be offered on the website, in the registration system and to appear on the diploma |
|
Implemented as of Fall 2023 |
| 5. Implement certificate programs | Certificated to be offered on the website, in the registration system and to appear on the diploma |
|
Fall 2023 |
| 6. Integrate the Joint Command and Staff Program (JCSP) into the MPA | Course-by-course analysis to be conducted |
|
Response updated four courses implemented for fall 2023 |
| 7. Improve tracking/monitoring of student progress | See recommendation 3 | MPA Chair | June 2024 (Updated as a part of the annual implementation update) |
| 8. Improve the post-course evaluation feedback mechanisms |
Program chair to meet with Director of QA to review past practices and explore new exit surveys. QA will brief students and faculty to promote higher response rates. |
|
Review of results in June 2024 |
| 9. Improved library support |
The programme chair will reach out to the Chief Librarian at Canadian Forces College (CFC) to understand how they provide library support at CFC. How-to” will be added to the yearly Orientation session and the instructions will be posted to the Moodle Graduate Orientation Page |
|
Fall 2023 |
| 10. Promote program better within CAF, DND and other related security organizations and the federal government | Discussion with the Registrar and Liaison to determine if there is a budget gap that prevents the promotion of Post Graduate Programmes in general and the MPA in particular. |
|
June 2024 |
| 11. Develop specific requirements for MPA 505 and MPA 507 | Requirements are developed and available. They will be presented to the students at the beginning of each semester |
|
Completed 2023 |
Conclusion
The ERC Report provided positive feedback on the outcomes of the MPA program. It confirmed that the program is academically strong and an asset to RMC, aligning well with its institutional goals and Canada's overall public administration learning infrastructure. However, the ERC identified areas with room for improvement, and RMC has taken steps to address the issues raised. RMC will continue to work toward program enhancement and improve student success in the MPA program.
The Dean of Graduate Studies, in consultation with the Dean of Social Science and Humanities and the Program Chair, is responsible for monitoring the Implementation Plan.