Performance Reporting

Performance Measure 1: Number of degrees, diplomas and certificates awarded

All students who meet the requirements of programs at public post-secondary institutions, including programs provided through the Industry Training Authority, are entitled to receive a formal qualification — either a degree, diploma, or certificate (including certificates of apprenticeship, qualification and Red Seal Certification). Counting these credentials provides an understandable and recognizable indication of the efficiency of the public post-secondary system, and reflects the system's capacity to meet student demand and to ensure sufficient numbers of graduates.

This measure is a count of credentials awarded (issued) by public post-secondary institutions, and the Industry Training and Apprenticeship Commission (now the Industry Training Authority), stated as a three-year average ending in the 2002/03 academic year.

The number of credentials awarded by post-secondary institutions depends on the number of students who complete and who apply for graduation from their education programs. Some of the factors that may affect the number of credentials issued by an institution include: historical and current institution capacity (based on physical capacity and the level of funding available); personal, social and economic factors that directly affect students' participation in and completion of their education program; and post-secondary institution business practices and enrolment management policies and procedures. While Ministry efforts to expand system capacity by increasing funding to public post-secondary institutions and increasing the targeted number of student FTEs may not directly produce comparable increases in credentials awarded immediately, by building capacity now, institutions will be able to increase the number of students who complete programs over the next few years.

To compensate for some of the effects on student program completion identified above, baseline and annual achievement data are calculated using a three-year average (i.e., the total for 2002/03 is an average of 2000/01, 2001/02 and 2002/03).

Results:

Total credentials awarded based on a three-year average ending in 2002/03 exceeded the previous year average by 3.2 per cent, and exceeded the targeted increase (1.5 per cent) by 1.7 per cent. This indicates that there was an expansion in the number of students who were able to complete and obtain qualification from their post-secondary education program. The following table provides a breakdown of the credential output from public post-secondary institutions in BC by credential type based on three-year averages ending in 2001/02 and 2002/03.

Performance Measure 2: BC public post-secondary graduate rate

This measure is referred to as a 'graduate rate' rather than a 'graduation rate' because it compares the number of graduates to the total British Columbia population aged 18-29 years, not to the number of students who actually entered public post-secondary institutions. This measure provides some relational context to the implications about system capacity inferred by the number of credentials awarded (See Performance Measure 1).

This measure compares credential data provided by public post-secondary institutions to data on the total British Columbia population cohort aged 18-29, which are obtained from BC Stats Population Extrapolation for Organization Planning with Less Error (P.E.O.P.L.E.). The 18-29 year-old cohort was selected as the most appropriate representation of post-secondary students in British Columbia. P.E.O.P.L.E. estimates are subject to annual revision, so baseline data may be restated in accordance to the most recent estimate for that year. Credential data are based on the actual number of credentials issued in the academic year.

Results:

Results for this measure indicate that the increase in credentials awarded for 2002/03 was a real increase as it exceeded the number expected simply due to the natural growth in the 18-29 year-old population in BC. This suggests that post-secondary institutions are contributing to an expansion in the proportion of population who obtained formal post-secondary qualification.

Performance Measure 3: Number of block and course transfer agreements

A distinguishing characteristic of the public post-secondary system in British Columbia is the amount of articulation between institutions, which enables students to transfer between institutions and receive credit for courses and programs completed at one institution when pursuing further education at another institution. The number of transfer agreements provides one indication of the efficiency and integration of the public post-secondary system.

This measure is a count of the total active transfer agreements registered with the British Columbia Council on Admissions and Transfer (BCCAT) on March 31. BCCAT is a publicly funded agency responsible for facilitating admission, articulation and transfer agreements between British Columbia public post-secondary institutions. The measure includes both course-by-course transfer and block transfer, which occurs when a group of courses, often in the form of a certificate or a diploma, is recognized for transfer credit.

This measure can provide useful information about the level of opportunity for transfer within the post-secondary system. However, it should be noted that the risks associated with achieving targeted performance include the number of block and course transfer agreements that may be affected by program changes at either sending or receiving institutions, or by reviews of agreements after a specified term. Also, a single new block transfer agreement may replace a number of existing course-to-course agreements, which would improve transfer opportunities for students but may be perceived as a negative result from the perspective of this measure. Finally, information for this measure may be incomplete if institutions do not report all of the transfer agreements they have negotiated. Course and block transfer agreement numbers reported by BCCAT are based on a snapshot of the number of agreements registered with BCCAT on a single day of the year (March 31) and do not reflect the ongoing additions and deletions of agreements.

Results:

The observed increase in course to course transfer agreements and block transfer arrangements indicate that there was an overall general increase in transfer opportunities for students at BC post-secondary institutions. The following table provides the number of agreements registered at BCCAT annually since 2001/02.

In light of some of the shortcomings associated with this measure noted above, the Ministry will continue to review its efficacy as an indicator of the performance of the transfer system for Service Planning and accountability purposes. In 2004/05 an additional measure will be developed to indicate the performance of the transfer system by measuring transfer volume.

Performance Measure 4: Student satisfaction with transfer

Credit transfer processes were established to facilitate smooth transition of students enrolled in courses/programs at one institution into courses/programs at another institution. This transition involves not only articulation agreements, but also those products and services within the public post-secondary system that inform students of the options available to them and how to take advantage of those options. BCCAT works with public post-secondary institutions both to ensure that students are informed of transfer opportunities and to develop policies and procedures to facilitate transfer between institutions.

This measure is the percentage of former students who transferred to a different institution, expected to transfer credit, and indicated via survey that they were either satisfied or very satisfied with their overall transfer experience. It is intended to gauge the efficiency and integration of the public post-secondary system by assessing students' perspectives of how successfully the system facilitates transition.

This measure is subject to the risks inherent with statistical survey techniques (response bias, sampling error, etc.). Further, student outcomes survey data may unintentionally reflect student assessment of matters unrelated to the specific measure (institutional capacity, availability of courses and programs, etc.). To alleviate these risks, the Ministry participates with qualified staff at BC Stats, public institutions, and system organizations like BCCAT to administer and design student outcome survey projects. The Ministry is confident that the long history of undertaking these surveys, and the technical expertise of all participants, allows for a high level of confidence in the survey results.

Results:

While the annual variation from the 2003 survey falls outside the confidence interval (margin of error), this should not be regarded as a meaningful variation. Outcomes survey data are best evaluated over time to alleviate the effects of cyclical, contextual or other structural fluctuations including statistical variation. Over the last three survey years overall satisfaction has remained stable.

For 2004/05 and for subsequent years, the performance target for this and other measures that rely on outcomes survey data will be based on performance trends.

Performance Measure 5: Mandates, roles and responsibilities reviewed and updated

The post-secondary system is facing a changing social and economic context, and student demand continues to grow. There is a need to make the most of the significant investment in the post-secondary education system. Clarification of the overall organization of the system and each institution's role in it will support integration and help to promote a more efficient system.

The review of mandates, roles and responsibilities presented an opportunity to assess the existing public system, including its legislative framework, to determine how it could be updated and further strengthened. There have also been recent Government decisions about significant developments in the public post-secondary system, including 25,000 new student spaces by 2010, and new institutions in the Okanagan and Thompson-Cariboo. The resulting changes will ensure the post-secondary system is poised to respond to the challenges and opportunities before us.

Results:

The target of introduction of legislation by 2003/04 was an ambitious estimate of the time required to conduct the review and to develop legislation. Legislative changes relating to the new institutions were not introduced prior to the end of 2003/04. However, a number of legislative amendments were introduced and passed in the Spring Legislative Session under the College and Institute Amendment Act, 2004.

Performance Measure 6: Number of student spaces in ABE, ESL, and ASE programs

Maintaining access to developmental programs like ABE, ESL and ASE is a priority for the Ministry. Developmental programs are important avenues for adult learners to achieve their educational and employment goals. Some of these learners are disadvantaged and some are newcomers to Canada. Developmental programs offer them the opportunity to gain the skills necessary to participate fully in the BC economy and society.

This is a measure of the number of student FTEs delivered by public post-secondary institutions in developmental programs including Adult Basic Education (ABE), English as a Second Language (ESL) and Adult Special Education (ASE) in the fiscal year.

Achieving the target for this measure depends in part on those factors that may affect student enrolment in these programs, such as level of student demand, institutional enrolment management policies and procedures, student retention issues, and the method for calculating FTEs. Student demand may be affected by a number of factors such as the economy or changes in the policies of other ministries and governments.

Results:

The baseline of 2003/04 actual student FTEs for this measure was determined after considering a number of other possibilities including a three-year average of developmental FTEs produced by an institution. The baseline for this measure is an accurate representation of the level of ABE, ESL and ASE programs institutions are able to deliver in the current environment and allows a direct comparison to be made with the first year that the Ministry has set targets for these FTEs (2004/05).

In the 2003/04 Service Plan, the baseline and performance targets were intended to be reported for each program separately. However, on review it was determined that by measuring developmental programs in this way institutions would not have opportunity to adjust their program offering to meet changing student demand or to address other factors that may arise over time. As a result, the baseline and performance targets for this measure are based on an aggregate total of FTEs delivered in all ABE, ESL and ASE programs in the BC public post-secondary system. Institutions will, however, continue to report FTEs under each program area separately in their audited FTE reports.

For greater clarity, this measure has been reworded for 2004/05 to "Student spaces in developmental programs [Adult Basic Education (ABE), English as a Second Language (ESL) and Adult Special Education (ASE)]."

Performance Measure 7: Total number and per cent of student population that are Aboriginal

Increasing Aboriginal student participation in post-secondary education remains a key strategic priority for Government and BC public post-secondary institutions. This measure is intended to provide an indication of the actual number of Aboriginal students in the post-secondary system and their proportion relative to the total number of students enrolled.

This measure is the count of Aboriginal students and the per cent of total enrolled students that are Aboriginal in the public post-secondary system. The information used to establish this count is primarily based on public post-secondary institution registration and administration systems. Aboriginal students are provided an opportunity at time of registration to self-identify as persons of Aboriginal ancestry. For many reasons, not all Aboriginal students choose to self-identify to the institution. Also, there are currently no standards for how questions about Aboriginal ancestry are phrased on student application forms, or standards for how such questions are recorded in student registration systems at institutions. As a result, public post-secondary institution registration data may under-represent the actual number of Aboriginal learners enrolled. To compensate for this limitation, counts from the administrative records of post-secondary institutions are enhanced by adding those students who were identified on K-12 administrative records as being of Aboriginal ancestry. To ensure student privacy, Freedom of Information and Protection of Privacy Act guidelines were followed.

Results:

While this method of calculating Aboriginal participation may not provide complete representation of Aboriginal student participation in post-secondary education, it represents best efforts under current data limitations.

Performance Measure 8: Total student spaces

Measurement and reporting of student enrolment (student FTEs) has been a feature of accountability processes for the public post-secondary education system for many years. This measure reflects the Ministry's ongoing focus on monitoring the capacity of the public post-secondary system which, along with other factors such as population size and affordability, will affect students' ability to access post-secondary education.

This measure is calculated using audited FTE reports submitted to the Ministry by public post-secondary institutions. Factors that may affect the achievement of the target include: student demand; student retention issues; institution enrolment management policies and procedures; and institutional restructuring and the development of a new industry training model.

Results:

Results for this measure exceeded the target, indicating that the total number of FTEs delivered by public post-secondary institutions exceeded the targeted total by 636. This resulted in an overall utilization rate 100.4 per cent.

Historical Data/Utilization Rates

Performance Measure 9: Per cent of annual educational activity occurring between May and August

This measure reflects the Ministry's commitment to maximizing utilization of institution campus facilities for instructional activity on a year-round basis. It compares the volume of instructional activity between May and August with that between September and April. The rationale is that increasing summer use of facilities for instruction will indicate increased year-round capacity and more efficient use of resources.

Public post-secondary institutions are required to demonstrate a measurable trend towards achieving year-round utilization of existing education facilities at the threshold level before capital expenditure requests to increase their physical capacity will be considered. Obvious constraints to this are the need for many students to work for a portion of the year to help finance their education and the need for periods of time to conduct major maintenance of buildings.

Results:

When interpreting the results for this measure, it should be noted that a significant proportion of each institution's facilities are used for non-instructional activity, and that some institutions are limited in their ability to expand summer use of facilities cost effectively. Nonetheless, this measure provides useful information for capital planning and offers an indication of efforts to increase efficiency.

Performance Measure 10: Number of private degree programs

The Ministry's commitment to increasing access and choice for BC post-secondary students resulted in the Degree Authorization Act (DAA). The DAA was specifically intended to expand degree opportunities for BC students by facilitating an increase in the number of institutions able to offer degree programs in BC.

Previously, private and out-of-province public institutions wanting to grant BC degrees needed to obtain authority through a special act of the Legislature. Under the DAA, special legislation is not required. Ministerial consent is granted only when an application has undergone a successful quality assessment process conducted by the Degree Quality Assessment Board and specific requirements have been satisfied regarding the posting of financial security to protect unearned student tuition revenues and to maintain access to transcripts in the event of institutional closure.

This measure counts the number of degree programs from private post-secondary institutions (in-province and out-of-province) that have received Ministerial consent in the current fiscal year under the DAA.

This measure can provide useful information about the range of degree options available in the province, but it should be noted that there are a number of factors that may affect performance in this area. The Ministry has no control over the number of applications received or the quality of those applications. Also, in addition to the number of programs approved, there will generally be a number of applications under review, and some rejected or withdrawn. Finally, the measure does not capture applications from out-of-province public institutions which have received Ministerial consent.

Results:

The DAA came into force on November 7, 2003. No applications from private post-secondary institutions had received Ministerial consent by the end of fiscal 2003/04, although a number of applications had been received and were being reviewed. As a result, the baseline for this measure was established based on the period from November 2003 to March 2004.

To reflect the scope of efforts to increase the number and range of degree program opportunities through the DAA more accurately, this measure has been reworded in 2004/05 to "Number of private and out-of-province public degree programs approved."

Performance Measure 11: Number of applied degree programs at public institutions

In 2003 the Ministry undertook additional initiatives to enhance student choice by expanding student access to degree level programs in regions of the province where they were not previously available. With the May 2003 consequential amendments section of the Degree Authorization Act (DAA) and with amendments to the College and Institute Act and the Institute of Technology Act all British Columbia's public colleges, university colleges and institutes were granted the ability to apply to offer applied degree programs.

This is a measure of the number of applied degrees authorized by these legislative amendments approved by the Minister in the fiscal year.

There are a number of factors that may affect performance on this measure. The Ministry has no control over the number of applications received. In addition, applied baccalaureate degrees offered by colleges and applied masters degrees offered by university colleges and provincial institutes are new entities. Alberta and Ontario's experience in delivering similar programs suggests that at some point the number of new applied degrees will plateau and further increases may not be expected.

Results:

The 2003/2004 baseline data reflects four applied degrees previously offered through a Capilano College/Open University partnership that were 'grand-parented' by the Ministry in May 2003.

Performance Measure 12: Number of student spaces in online learning programs

Online learning programs provide increased access, flexibility and choice for students pursuing post-secondary education. The strategy to expand online learning opportunities enables students to obtain access to online courses/programs that are offered from any institution, thereby enhancing the selection of courses and programs available.

This measure consists of student FTEs in online programs at public post-secondary institutions. It is an indicator of the number of students accessing educational opportunities online. Programs considered to be online for this measure are those that are offered wholly online or by distance, and which lead to an online or distance program credential, and are available to any qualified student within the province.

Factors that may affect the achievement of targets include: student demand; student retention issues; institution enrolment management policies and procedures; and administration practices in counting FTEs for these programs.

Results:

For online learning, where part-time or single course enrolments are common, counts of FTEs typically represent more separate enrolments and individual students taught than counts of FTEs in other programs.

For greater clarity, the name of this measure has been reworded for 2004/05 to "Student spaces in online learning (BCcampus) programs."

Historical Data/Utilization Rates:

Performance Measure 13: Student outcomes — skills gained

One of the expected outcomes of a quality post-secondary education is that students acquire the generic skills and knowledge necessary to participate in the labour market and to engage in social, cultural, and political life. This measure provides an indication of how well this outcome is realized. It is the percentage of former students who, when surveyed, indicated that their education helped them develop specific, generic skills commonly regarded as necessary to lead a productive life and that are similar to the types of skills deemed necessary for successful employment by the business community and the Conference Board of Canada. As well, the skill categories included here represent the key core skills post-secondary institutions attempt to impart to their students and which are commonly associated with a quality post-secondary education.

While this measure can provide useful information, skill gain is difficult to quantify because there are seldom mechanisms to measure skill level prior to admission. Also, different educational programs emphasize different skills, and specific cognitive and manual skills may not be covered by the list of skills used in this measure.

See the context for Performance Measure 4 for discussion of factors that may affect performance associated with measures of survey data.

Performance for all skill categories for college, university college and institute former students either maintained or increased from the previous survey year within survey margin of error. This indicates that students continue to value the quality of education they receive from BC public post-secondary institutions.

Performance Measure 14: Student satisfaction with education

Students' assessment of how satisfied they were with their education experience is an understood and accepted measure of quality.

This is a measure of the percentage of former post-secondary students who were very satisfied or satisfied with their post-secondary education.

While it is acknowledged that there are a number of factors beyond the control of public post-secondary institutions that may influence student satisfaction, it expected that these factors are fairly distributed across the group of student respondents in any given year, and are relatively consistent over time. Therefore, the unique characteristics of the group of student respondents is believed to be less of a factor in their satisfaction assessment than the quality of education they received.

See the context for Performance Measure 4 for discussion of risks associated with measures based on outcomes survey data.

Results:

Performance results for this measure indicate that students continue to express high satisfaction with their education experience at BC public post-secondary institutions and of the quality of education they received.

Performance Measure 15: Student satisfaction with quality of instruction

Students' assessment of the quality of the instruction they received in their education programs provides a strong indication of the overall quality of education delivered at public post-secondary institutions in BC.

This is a measure of the percentage of former post-secondary students who rated the quality of instruction in their education program as very good or good.

See the context for Performance Measure 4 for discussion of factors associated with measures based on student outcomes survey data.

Results:

Performance results for this measure indicate that students continue to rate highly the quality of instruction they receive in their programs of study at BC public post-secondary institutions.

Performance Measure 16: Student assessment of usefulness of knowledge and skills in performing job

Students' assessment of the knowledge and skills they gained in their program of study in relation to the requirements of their subsequent employment reflects the relevance of the public post-secondary education system to immediate employment.

This is a measure of the percentage of university graduates and college and institute former students who indicated the knowledge and skills they acquired through their education was very useful or somewhat useful in performing their job. Responses from students not employed when surveyed are not included in the results.

See the context for Performance Measure 4 for discussion of factors associated with measures based on student outcomes survey data.

Results:

Students continue to indicate that the knowledge and skills they acquire are useful in their subsequent employment. This provides some indication of the continuing high relevance of post-secondary education in BC.

Performance Measure 17: Student outcomes — unemployment rate

This measure provides some indication of how successful former students were in making the transition from post-secondary education to the labour market, and therefore to some extent reflects on the relevance of their post-secondary education to the needs of the economy. Students who do not pursue further post-secondary education will move to the labour force with the expectation that their education and training will enable them to find employment related to their education within a reasonable period of time.

This is a measure of the number of university graduates and college and institute former students who were unemployed in the reference period prior to time of survey.

See the context for Performance Measure 4 for discussion of factors associated with measures based on student outcomes survey data. Limitations of this measure include the number of factors, in addition to the relevance or quality of post-secondary education received, that influence the labour market transition of former students including general economic conditions, geography (regional variations in employment), demographics (characteristics of student), and type of education program taken. Variations in unemployment rates over time are at least as likely to be influenced by general economic conditions as they are by the relevance or quality of post-secondary education former students received.

However, it is widely accepted that post-secondary education, to some extent, influences an individual's ability to obtain and maintain employment. Statistics Canada Labour Force Survey clearly show that unemployment rates for people with post-secondary education are significantly lower than for people without post-secondary education. In addition, research suggests that while individuals with post-secondary education are susceptible to changes in general economic conditions, they may experience the effects of such changes differently than individuals without post-secondary education.

Results:

It is important to note that these performance measure unemployment rates are based on survey responses from former students to the BC student outcomes surveys. These rates are compared to annual average unemployment rates from Statistics Canada monthly Labour Force Survey (LFS). One significant difference between the two surveys is that the majority of graduates and former students fall within the 18-29 years age range whereas the generally reported BC unemployment rate includes all individuals 15 years or older. The 2003 unemployment rate from the LFS for all British Columbians in this age group is 11.1 per cent compared to 8.1 per cent for those 15 years or older. Generally, as individuals enter the labour market and gain more job experience the likelihood of their finding and maintaining employment increases. Because former students from BC colleges, university colleges and institutes and baccalaureate graduates from BC universities are generally younger and newer 'entrants' to the labour market, their unemployment rate is higher in comparison to that for the total population. Therefore, a meaningful comparison is between former students and non-students within the same relative age group. With this type of comparison it is possible to attribute some of the difference between the groups unemployment rates to post-secondary education. The table below shows unemployment rates for comparable groups in BC.

Unemployment Rates:

The baseline was established for this measure. Future reporting will take into account those factors discussed above that will affect accurate interpretation of the performance results for this measure.

Performance Measure 18: Ratio of median debt to median income of graduates

The Ministry undertakes to maximize student access to a high quality, affordable and relevant post-secondary education system based on the resources available within the current economic environment. One of the key considerations in this process is determining the most reasonable and equitable balance between the costs (borne by students, taxpayers and institutions) and benefits (shared by employed individuals/former students, society/taxpayers, and institutions) of post-secondary education.

This measure was intended to address one aspect of this balance, that is, to reflect the cost impact on students (as indicated by student debt levels) in comparison to their ability to repay debt upon completion of their program (as indicated by the financial benefit/income students receive from their subsequent employment).

This is a measure of the ratio of median debt for students with debt at time of leaving their institution to their median income one year (college, university college and institute former students) to two years after graduating (university baccalaureate graduates.) At the time of the 2003/04 – 2005/06 Ministry Service Plan this measure was still under development and the performance target for 2003/04 target was to establish a baseline.

Results:

The actual performance results for 2003/04 identified above as the baseline for this measure represents the amount of debt for each employment dollar earned in one year. The table below provides the dollar values associated with this ratio.

In the course of developing a baseline it became apparent that this measure would not adequately reflect the intent of the objective for a number of reasons. First, this measure focuses only on those students who incur debt in order to obtain a post-secondary education, and does not adequately address the experience of over 50 per cent of students who do not incur debt. Second, the measure does not reflect the experience of students who may have been academically qualified but who could not afford to pursue their education. Third, this measure does not consider affordability from the perspective of taxpayers or public post-secondary institutions. That is, it does not consider what proportion of the full cost of post-secondary education is borne through Government funding and by funds earned or obtained by post-secondary institutions themselves. As a result, this measure was deleted from the 2004/05 Service Plan.

Information on student outcomes is available on an annual basis from the College and Institute Student Outcomes Survey project (www.outcomes.bcstats.gov.bc.ca) and from the University Baccalaureate Graduate Survey project (www.tupc.bc.ca).