EDUCATION: by Alan BarcanNews Weekly
New assessment can mean almost anything
, March 24, 2001
The move to outcomes-based assessment is now the vogue, but it has come at a price says Alan Barcan.
Outcomes-based education (sometimes called outcome-based education) is one of the numerous educational initiatives which have proliferated over the last two decades. Teachers and parents have had to grapple with a flood of confusing terminologies promising panaceas for the ills of schooling. The new jargon includes profiles, key learning areas, benchmarks, frameworks and competencies. Tasmania contributed "Key Intended Learning Outcomes", Victoria its "Curriculum and Standards Frameworks", and Western Australia its "Outcome Statements". Other systems had other variants.
In New South Wales, as elsewhere, outcomes-based education permeates the whole school system, from kindergarten to Year 12. In South Africa it is even compulsory in universities.
When my grandchild came home after his first day in a NSW state school he brought with him a pink sheet headed "This is your little learner's first Kindergarten Work Activity. It was completed on Wednesday 31st January, 2001 - the first day of Kindergarten." On this page he had completed two tasks - to write his name and to make a drawing titled "This is a picture of me at ..... Public School". The foot of the page carried a proud sticker, "Principal's Award", together with a printed statement.
* demonstrates a beginning understanding for learning to read and write;
* writes own name correctly using lower case letters;
* dictates a sentence which matches a picture.
In the distant past his work might well have carried a mark, such as 9 1/2 out of 10, with a comment "Very Good Work" or perhaps a gold star. But that was another era. We live in a new age, with new ideas. Often these are old ideas in new, more pretentious, dress.
At later stages of schooling parents receive half-yearly and yearly reports describing their offspring's progress in terms of a plethora of "outcomes".
Many of the new terms are linked with efforts to re-introduce greater control of curriculum content and standards after some two decades of minimal surveillance. For the first two-thirds of the 20th century the curriculum was held in place by prescribed or recommended subject syllabuses, externally-assessed public examinations, widely-accepted or prescribed textbooks, and systems of inspection.
School principals (or subject masters in large secondary schools) kept an eye on what was taught. In any case, teachers and the educational community shared a general consensus on what constituted appropriate school knowledge. From the late 1960s almost all of this disintegrated.
Today examinations survive only at the end of Year 12 (and not at all in Queensland or the Australian Capital Territory). In recent years externally-controlled literacy and numeracy tests have been introduced, usually in Years 3 and 5 of primary schools and sometimes in Years 7 or 10 of the junior secondary school.
Outcomes-based education had a very simple origin. In the 1980s some commentators on education began to argue that inputs (specifically funding) had received too much attention to the neglect of outcomes (student achievement).
As is the way in modern education, the new bandwagon was given a theoretical gloss by American educationists. William G. Spady, Director of the High Success Network in Colorado, USA, gained attention with his 1988 journal article "Organising for Results: the basis of authentic restructuring and reform". He followed this up in 1991 with a jointly-written article, "Beyond traditional outcome-based education".
Spady presented a quite uncontentious definition of his program:
"Outcome-based education means focusing and organising a school's entire programs and instructional efforts around the clearly defined outcomes we want all students to demonstrate when they leave school."
Who could object to this? But controversy lay in the details.
When Australia embarked on a flurry of educational reform between 1987-1993, conservatives welcomed OBE as a healthy antidote to the view that what was important was process, not content and that the point of the curriculum was not to impart knowledge (there was "too much knowledge") but to develop skills, such as problem-solving. On the other hand, radicals and progressives saw OBE as a welcome alternative to external tests or examinations.
In September 1992, Spady conducted a series of workshops and lectures in Canberra, Sydney, Brisbane, and Melbourne. But he gave the game away when he identified three forms of outcome-based education:
* Traditional, which emphasised subject-matter content;
* Transitional, which assisted "the cultivation and integration of higher order competencies" (e.g. critical thinking, problem solving, effective communication skills); and
* Transformational, which emphasised complex role performance "in authentic contexts" (e.g. success after school). It was possible to smuggle a variety of things into "outcomes".
Not everybody was impressed by the new fashions. The chairman of the NSW Board of Studies boldly remarked that as far as he could see "aims, purposes, functions, objectives, outcomes and pointers are all the same thing".
His Minister had other ideas. He was dismissed in March 1994.
The Eltis "Review of Outcomes and Profiles in NSW Schooling", presented to the Minister for Education in 1995, located nearly 3000 items of world research on outcomes-based education. An inspection of 300 of these showed that most simply provided a description of this approach; others provided arguments for and against it; few demonstrated support for its use beyond statements about improved test performance or better attitudes to learning or increased accountability.
Nonetheless, the rolling bandwagon had gathered momentum. When, between 1990 and 1993, the Commonwealth Government, working through the Australian Education Council (the Ministers for Education of the Commonwealth, the six States and two Territories) sought to produce a National Curriculum based on "Statements" and "Profiles", the Profiles adopted an outcomes-based approach. Even though the National Curriculum collapsed in July 1993, when the AEC referred the scheme to the States and Territories for further consideration. These authorities ultimately accepted many features of the proposed Profiles, including outcomes-based education.
The NSW Education Reform Act of 1990 required that syllabuses endorsed by the Board of Studies indicate "the aims, objectives and desired outcomes in terms of knowledge and skills that should be acquired by the children at various levels of achievement ...".
The Board issued a document, Curriculum Outcomes, in September 1991 which defined outcomes as "intended results of teaching and learning" expressed as a set of broad, comprehensive, assessable and observable indicators or benchmarks of student achievement". The key word here was "assessable", for education reform in New South Wales included strengthening of syllabuses, testing and examining.
The document took, as an example, the objective "to develop skills in critical thinking". It suggested six outcomes, such as "distinguishes between fact and opinion in newspaper articles"; "distinguishes between relevant and irrelevant information in historical source material"; and "distinguishes fallacious reasoning in a speech". Difficult capacities, even for a teacher!
But when new syllabuses started to appear, the program had become quite complex. For instance, the syllabus for Community and Family Studies for Stage 6 (Years 11 and 12) presents teachers with seven Objectives, which generate 17 "Preliminary Outcomes" (for Year 11) and 18 "HSC Outcomes" (for Year 12). Now how does an examiner assess the first HSC Outcome: "analyses the effect of resource management on the wellbeing of individuals, groups, families and communities"? This could be extremely complex, or extremely simple! And, indeed, the scope for interpretation is one reason for the widespread acceptance of outcomes-based education.
To further complicate matters, the syllabus also requires attention to "key competencies", related to various skills. The pre-history of Outcomes
One benefit of studying the history of education is the discovery that there is little that is really new in education, though technological innovation does occasionally bring deep change. Precursors of outcomes appeared in the 1980s. One was a movement to restore attention to content through criterion-based teaching and assessment.
Many radical and progressive educationists opposed assessment, but if it was to be used they favoured norm-based assessment. This simply ranks students, relative to their fellow-students. But it reveals nothing about what has been learnt or, indeed, whether anything has been learnt. No matter how badly standards might slump, the same percentage of students will pass, or fail, or gain high distinctions.
In Victoria, the 1985 Blackburn Review on Post-compulsory Schooling refuelled the long-going argument over education; its proposals included a new credential, the Victorian Certificate of Education. Bill Hannan argued (Assessment and Evaluation in Schooling, 1985) for "work-based" or goal-based assessment. Hannan, a prominent leader in the radical Victorian Secondary Teachers' Association, was about to start his "long march through the institutions"; he became chairman of the State Board of Education in May 1986. Hannan's view of goal-based assessment was that the goals to be achieved should be stated in advance. If the work was done as specified, the goal was achieved. The assessment could be "satisfactory completion". Hannan also said that "the fundamental idea is that participating students should succeed".
Criterion-referenced assessment is based on the question: Has the student met certain defined criteria or goals? The assessment indicates how the student's work compares with someone's expectation, expressed as performance criteria, of what students at that year level should be able to do. The assessment might be expressed as grades, or might not. It could accommodate goal-base assessment.
About 1988 a new term came into favour, standards-based assessment. This term also originated in America.
In all cases the crucial question is who sets the standards, criteria, goals - or outcomes. And who decides whether they have been met? The door is wide open to almost any sort of interpretation of what constitutes mastery of content.
The new HSC in New South Wales is to be standards-referenced, not norm-referenced. At the end of this year the HSC examiners will face a moment of truth. If standards are firmly applied, one suspects that many candidates will fail. This involves what Sir Humphrey of Yes Minister was prone to call "a courageous decision".
If you are confused by all this, don't worry. Confusion is part of the assessment game played by modern educationists. And why? So that parents or politicians will not be upset by the naked, objective, revelation of standards? So that employers will not too hastily reject under-qualified applicants for a job? So that teachers will not be embarrassed by the poor results of their teaching? And so that some students and some parents will not be upset by reports suggesting low levels of achievement?
The NSW Board of Studies booklet, Curriculum Outcomes, listed 10 ways in which statements of outcomes can assist teachers. Assuming they are listed roughly in order of importance the first four are:
* inviting focus upon the product as well as the process of teaching;
* providing specific guidance for planning the learning environment;
* providing a focus for assessment; and
* defining the content level of the syllabus more precisely.
The final advantage was "giving parents, employers and the wider community a clearer understanding of the instructional intent and likely achievements of students".Some Problems
In his 1995 Review, Eltis found that primary teachers were more positive about the outcomes approach than their secondary colleagues. "Both were equally concerned about workload, assessment and reporting issues and time taken from teaching, but secondary teachers also focused on subject-related concerns. Primary teachers worried about having to assess in six key learning areas while secondary teachers worried about the number of students they had to assess in relatively few periods." Eltis concluded that considerable confusion remained about the terminology and language in Outcomes and Profiles.
In 1998 participants in a project conducted by the Australian Curriculum Studies Association identified some contentious aspects of outcomes-based education. (1) differing conceptions of an outcome-based approach to education, revealing variation in underlying assumptions and beliefs; (2) different meanings attributed to commonly used terms; (3) differing views whether OBE should be achieved by gradual evolution or by rapid revolution; (4) uncertainty whether successful implementation of OBE in a school needs a set of pre-conditions, such as motivation, co-operation, resources; (5) uncertainty whether OBE should apply to some or all levels of schooling.
Outcomes-based education is a multifaceted concept. It can cater for genuine academic work or low-level, activity-centred, mindless education.
Therefore it exerts an initial appeal to a wide range of contemporary educational philosophies or ideologies.
Its character depends heavily on who operates the program.
As developed by educational theorists, it can become so elaborate as to cause confusion amongst teachers and parents and employers. Its complexities can produce confusion and misunderstanding.
It can lead to onerous pressures on teachers.
The older style of tests and exams give a clearer and simpler definition of the curriculum.
Most parents seeking information about their child's academic progress still look for two old-fashioned items - a mark (out of 10 or 100) or a grade (A, B, C or High Distinction, Distinction, Credit, Pass) which suggests an absolute standard; and place in class, which suggests a relative or comparative standard. Or in educationese, "a combination of criterion-referenced and norm- referenced assessment".