Outcomes Based Education

Whenever I start a new job or project, I like to reflect on this statement, "Begin with the end in mind".  It comes from Steven Covey's "7 Habits of Highly Effective People".

Its based on the concept that all things need to be created twice. 

1. You need to create it mentally in your mind; 
2. You create it again physically.

The notion is that for a project to be successful, you need to start with a vision, something to align yourself to. If its something selfless and community focused, your team mates can align to it too. When you create a picture in your mind of the end goal, you get clarity and a sense of purpose, and then setbacks don't really matter, because it just means you need to find a different way to reach that outcome. Once you achieve that outcome, you can move on to the next level.

This is the concept that outcomes based education (OBE) is based on and this is what I want to cover with you today. To further understand OBE, lets compare it with Traditional Education.

There's a Ted talk on YouTube where Sal Khan (from Khan Academy) gives a talk called, "Lets teach for mastery, not test scores".

When an employer hires a graduate fresh out of college, what do you think they value more - the graduate's proven competencies or the graduate's grades? Employers don't really care about grades. The language they really care about is skills and competencies. Let's think about this using the analogy Sal gives in his talk.

Say you're building a house. The contractor says "OK we have two weeks to build the foundation.  Maybe it rains, maybe the concrete is still wet, maybe the building is not up to code. The inspector takes a look and says...well...I'll give it an 80%". You say, "Great thats a B, lets go ahead and build the first floor".

Imagine if you built a house based on grades. That would be absurd. The house would collapse in the end! The focus needs to be on getting the students to master the outcomes (i.e., skills and competencies) set for the program.

As a program coordinator in a Higher Education College program, your ultimate end goal is to help students reach their career dreams and become work ready. But on a more granular level, there are already a set of program outcomes prescribed for you.

It stands for Vocational Learning Outcomes. Did you know that every program here at NCT, has a set of VLO's (or program level outcomes) that are mandated by the ministry? Lets pick one program here at NCT, and we'll take a look at what the VLO's are.

1. Business General
2. Business Sales and Marketing
3. Business Accounting
4. Computer Programming
5. Hospitality and Tourism Management
6. Human Resource Management
7. International Business Management

It's important not to confuse program outcomes with course outcomes. You're probably all aware of course outcomes, because as an instructor at NCT, one of the first things you would have been given to prepare for your class is probably the course outline. On the course outline, you would have seen a list of course outcomes, which are essentialy the things the students should know and be able to do by the end of the course. But as a course instructor, you also need to be aware of the program level outcomes (or VLOs), and how your course is contributing to the overall goals of the program. Let's take a look at some.

1. Identify and discuss the impact of global issues on an organization’s business opportunities by using an environmental scan (competency=analysis).
2. Apply principles of corporate sustainability*, corporate social responsibility and ethics to support an organization’s business initiatives (competency=social responsiblity).
3. Use current concepts/systems and technologies to support an organization's business initiatives (competency=enterprise software)
4. Apply basic research skills to support business decision making (competency=research)
5. Support the planning, implementation and monitoring of projects (competency=project management)
6. Perform work in compliance with relevant statutes, regulations and business practices (competency=compliance)
7. Explain the role of the human resource function and its impact on an organization (competency=human resources)
8. Use accounting and financial principles to support the operations of an organization (competency=accounting)
9. Describe and apply marketing and sales concepts used to support the operations of an organization (competency=marketing)
10. Outline principles of supply chain management and operations management (competency=supply chain managment)
11. Outline and assess the components of a business plan (competency=business planning)
12. Develop strategies for ongoing personal and professional development to enhance work performance in the business field (competency=leadership and professional development).

Not only are these vocational learning outcomes (i.e., program level outcomes) mandated by the ministry, but they also are the same outcomes that are advertised to your students through your program's web page. We are essentially saying to them, "When you take the Business General program, these are the things you will KNOW and be able to DO when you graduate". These are the competencies you will be able to put on your resume. It's a promise that we've made to our students. So how do we ensure we're living up to our promise (More on this later)?

Another interesting thing about VLOs-the ministry does not approve VLO's unless they reflect what is needed in the industry. When crafting the VLO's, they are intentional about having the language reflect the same language you might see in a job description.

These vocational learning outcomes are the END IN MIND that you need to picture and strive for at all times. What you choose to put in your lesson plan, your teaching and learning activities, and your assessment methods, all need to align and contribute to at least one of these goals. So let's talk about how we can use Bloom's Taxonomy as a tool for alignment.

What is Bloom's Taxonomy?

Its a classification system which helps you quickly identify the level of learning expected in a course or program outcome, based on the verbs used.

The framework identifies 6 levels of cognitive learning (read from bottom up):

Using Blooms Taxonomy Verb charts can help in identifying what level of learning is expected in a particular VLO:

1. Each person in the room will be given a verb on a sticky note
2. At the front of the class you will see labels for the six levels of learning based on Bloom's Taxonomy. 
3. Place your verb under the Bloom category you think it belongs under.
4. Look at the Results, Discuss, Move verbs to the correct column as necessary.

1. Now each person will get a Business-General VLO (one of the 12 VLO's) on a sticky note. 
2. Place the sticky note under the Bloom level you think is most appropriate.
3. Look at the Results, Discuss, Move assessment methods to the correct column as necessary.

1. Now each person is going to get an learning activity on a sticky note.
2. Place the Learning Activity under the Bloom category you this is most appropriate
3. Look at the Results, Discuss, Move learning activities to the correct column as necessary.

1. Now each person is going to get an assessment method on a sticky note.
2. Place the assessment method under the Bloom category you this is most appropriate
3. Look at the Results, Discuss, Move assessment methods to the correct column as necessary.

Based on the level of learning identified by Bloom's Taxonomy, you can get recommendations on appropriate learning activities and assessments to use in your course:

Grab a coffee or water. When we return, we'll talk about Curriculum Mapping, OQF, POI Framework, Program Review and CQAAP

This is where Curriculum Mapping becomes important.

If any of you are Course Leads for the Business General program than this is all going to sound familiar to you because we talked about it in our workshop yesterday.

Curriculum mapping allows us to have a wholistic look at all the courses in the program to see which courses are contributing to the overall goals of the program, how much coverage is happening, at what level of complexity, and whether there are gaps and redundancies that need to be addressed.

It also allows to see if our teaching and assessment methods align with the level of complexity that the outcomes are written.

Let's take a look at a completed set of curriculum maps.

This particular map, based on the International Business Management Program at NCC, has 6 layers of mapping visualizations.  The first is the Level of Inclusion Map.

It is  a high level map that gives us a sense of general coverage. It answers the questions:

• Are we meeting all the VLO's mandated by the ministry? 
• Is there a VLO that is not met by at least 2 courses in the program? Are there any gaps?
• Is there a course that is not contributing to at least one VLO in the program?
• Are we adequately assessing students against each VLO in the program?

The second map is the Level of Complexity map.

It answers the questions:

• Are students reaching "mastery" or "proficiency" in each VLO by the end of the program? (remember Sal Khan's message - let's teach for mastery not test scores? This is where we see if students are reaching mastery.
• Are there courses sequenced correctly. Is there a natural progression of learning happening where concepts are introduced, then reinforced, than meeting proficiency?

The third map is the Level of Emphasis map. 

It answers the questions:

• Is there too much redundancy in the program? 
• are there multiple courses that are covering a concept strongly? Is there an opportunity to take some of the redundancy out and replace it with fresh content on emerging trends in the industry?

The fourth map is the Teaching Method map.

It answers the questions:

• are we using authentic teaching methods appropriate for the level that the outcome is written? This is where Bloom's Taxonomy becomes useful. 

The fifth map is the Assessment Method map. 

It answers the questions:

• Are we using authentic assessment methods appropriate for the level that the outcome is written? (Again Bloom's taxonomy is useful here)
• How much experiential learning is happening in the program?
• Is there a good balance between group work and individual work across the program?

The sixth map is the Essential Employability Skills (EES) map. More on EES's in the next section.

The Curriculum map is the main piece of evidence for the program review self-study. If your program is scheduled for a program review (i.e., the Business-General program this year), then most of the recommendations for improvement related to your teaching and assessment practices, will come from the curriculum map.

All Ontario college graduates holding the credentials of an Ontario College Certificate, Ontario College Diploma, an Ontario College Advanced Diploma, or Ontario College Graduate Certificate (or the French-language equivalent) must be able to reliably demonstrate the EES required in each of these six skill categories:

1. communication skills outcomes:

• communicate clearly, concisely and correctly in the written, spoken, and visual form that fulfills the purpose and meets the needs of the audience
• respond to written, spoken, or visual messages in a manner that ensures effective communication

2. numeracy skills outcomes:

• execute mathematical operations accurately

3. critical thinking and problem solving skills outcomes:

• apply a systematic approach to solve problems
• use a variety of thinking skills to anticipate and solve problems

4. information management

• locate, select, organize, and document information using appropriate technology and information systems
• analyze, evaluate, and apply relevant information from a variety of sources

5. interpersonal

• show respect for the diverse opinions, values, belief systems, and contributions of others
• interact with others in groups or teams in ways that contribute to effective working relationships and the achievement of goals

6. personal

• manage the use of time and other resources to complete projects
• take responsibility for one's own actions, decisions, and consequences

The Ontario Qualifications framework houses specifications of the knowledge and skills (learning outcomes) which each credential or qualification (for example, certificate, diploma, bachelor, masters, or doctoral degree) is intended to achieve.

In 2002, an Ontario government agency, the Postsecondary Education Quality Assessment Board (PEQAB), created by the Post-secondary Education Choice and Excellence Act, 2000, introduced the first qualifications framework in Canada. The Ontario Qualifications Framework (OQF) is the first complete framework in Canada, and it details a full range of postsecondary credentials including apprenticeship certificates. The OQF specifies the qualifications offered by Indigenous Institutes, the qualifications for Private Career Colleges, the qualifications awarded by Colleges of Applied Arts and Technology (CAATs) and those offered by universities and other authorized providers.

The Framework for Programs of Instruction Minister's Binding Policy Directive defines the ministry's expectations for the roles and responsibilities of colleges regarding awarding of credentials. More specifically, it highlights the complexity of courses that are required in the program by credential level (see page 10 of the following document for the summary table).

It also specifies which credentials mandate coverage of Essential Employability Skills Outcomes.

A cleaner version of MCU's Framework for Programs of Instruction with a few additional requirement made specifically for NCC programs.

At Niagara College, program review is the systematic collection and analysis of data and other program information that leads to continuous program improvement. Program review at NCC occurs on a 6 year cycle. This means that every academic program at NCC will go through a comprehensive program review process every 6 years.

Even though Faculty are constantly “reviewing” their program, the cyclical program review process makes these informal activities more systematic, more public and allows connections to be made across data sets and other information sources.

There are several benefits of program review including:

• Opportunity to take a holistic approach when reflecting on your program.
• Engage in in more productive conversations about the status of student success and make sound, rational decision about how it can be improved.
• Understanding what is working well, and what can be improved.
• Ensure stakeholder voices are captured, and that key college stakeholders have an understanding of strengths, challenges and opportunities of your program.
  

Every Niagara College Post –Secondary program follows a six-year program review cycle. Year one of the cycle starts the process, and will be the focus of the next 15 months. In years two and three programs are focusing on implementing and achieving their action plan that is developed at the end of year one. In year four, programs write a short report that is a ‘check in” on action plan implementation as well as a brief analysis of new data and any changes in industry that may impact the program. If a program follows a Program Description (as opposed to a Program Standard) year five of the cycle typically has programs review the Vocational Learning Outcomes (VLOs) of the program to ensure that the language used it current and aligns with expectations of industry. In year six, the focus is on updating the program curriculum map to get ready to start the process again.

CQAAP is an institutional level process that involves the regular and cyclical review of each college’s quality assurance mechanisms. While program level reviews ocurr on a 6 year cycle, CQAAP, the institutional level review occurs on a 5 year cycle.

There are 6 CQAAP standards to reflect in the CQAAP self-study, which provide the framework for Ontario’s colleges in assessing the extent to which their quality assurance mechanisms meet the established standards. Its purpose is developmental and its intent is to ensure continual improvement.

The standards are listed on page 5 and 6 of the following presentation deck, which was used to introduce NCC Administrators to CQAAP during the last audit in 2021/22.

The standard that NCT faculty would be most interested in is Standard 4 (see page 10 of the PPT deck):

• Effective quality Assurance Mechanisms ensure the quality of program delivery

as it directly relates to how instructors are conducting classes, assessing students, and ensuring VLO's are met. Standard 4.4 stresses the importance of connecting courses to VLOs.