Dr Petra Lietz

Faculty of Business and Law, Central Queensland University

International Educational Consultant, Hamburg, Germany

Ms Danielle Helbers

Faculty of Education and Creative Arts, Central Queensland University

Paper presented at the Australian Association for Research in Education (AARE) annual meeting, Adelaide, 29 November - 3 December, 1998

Dr. Petra Lietz, Associate Dean (Research)

Faculty of Business and Law, Central Queensland University,

Rockhampton, Qld 4702, e-mail: p.lietz@cqu.edu.au

Dr Dieter Kotte, Ückerstraße 38, 22547 Hamburg, Germany,

e-mail: dieter_kotte@magicvillage.de

The 'Asian Crisis' has demonstrated how easily Australia can be affected by global financial events. The implications are twofold: First, Australia cannot afford to become complacent about developing adequately trained human resources. Second, the globalization of trade requires a sound understanding of complex Economics issues to appreciate the implications of certain developments.

These implications emphasise the need for school leavers in Australia and elsewhere to have acquired an adequate knowledge of economic issues. Currently, Economics is an elective school subject in all States and Territories. In Queensland, only students in Year 11 and 12 have the possibility to study Economics.

The major aim of the Economic Literacy Survey - Queensland 1998 was to assess Economic literacy levels of students at the senior secondary school throughout Queensland. The target group, thus, comprised students from both Year 11 and 12.

The Economic Literacy Survey built on last year's Economic Literacy Pilot Study (Lietz & Kotte 1997) conducted in schools throughout Central Queensland. Piloting served to validate the various questionnaires for students, teachers and schools participating in the survey. Performance in Economic Literacy was assessed using the Australian Test of Economic Literacy (AUSTEL-11) which was adapted from the Test of Economic Literacy (TEL) developed by the Joint Council of Economics Education in the United States (Soper & Walstad 1987). Students were requested to answer multiple-choice test items each offering four possible answers of which only one was correct. In addition, background information was collected from students, teachers and school principals. This wealth of data allows for the development of complex hierarchical models explaining outcome differences among students and schools.

A second important aim of the Economic Literacy Survey was to investigate and develop new ways of performing large-scale student assessments. For the first time ever in a project of this magnitude schools had the choice to undertake the same assessment by a test which could be answered using either paper & pencil, personal computer (PC) or the World Wide Web (WWW).

This paper describes basic performance differences in economic literacy across Queensland's senior secondary students. In addition, several interesting findings based on responses to the Student Background Questionnaire, the Teacher Questionnaire as well as the School Questionnaire are also presented. Special emphasis will be given on the procedures and conduct of the PC- and web-based test options. Finally, remarks will be made about the positioning of the Economic Literacy Survey in an attempt to standardize Economics assessment as part of the OECD educational indicators policy.

The Economic Literacy Survey was conducted between August and September 1998 at Year 11 and Year 12 level in schools across Queensland. Participation was voluntary. In total, nearly 1400 students enrolled in Economics from 74 schools took part in the study. This represented nearly one third of all schools offering Economics at senior secondary school level. Hence, the following results are of relevance to the educational system in Queensland.

Schools were contacted by phone and invited to participate. For the purpose of the survey a coordination center was set up at the Faculty of Business and Law at Central Queensland University. As a novelty in large-scale school assessment schools were given the choice of testing by paper & pencil, PC or the World Wide Web (WWW).

In addition to tests assessing Economic Literacy, students were also asked to complete a background questionnaire which, in general, took no more than 10 minutes to complete. Questions covered, for example, age, gender, books at home, parental educational level, leisure time activities, spending habits and attitudes towards school in general and Economics in particular. Furthermore, teachers and school principals or deputies were asked to complete separate background questionnaires. Here, information was gathered on teaching experience and qualification, instructional practices, school size and resources as well as enrolment figures in Economics.

In this section, some descriptive statistics of student background variables from the study of Economic Literacy are presented. As mentioned earlier, students were asked to complete a background questionnaire which, in general, took approximately 10 minutes to complete. Questions covered, for example, age, gender, books at home, parental educational level, leisure time activities, spending habits and attitudes towards school in general and Economics in particular. Such background information can provide insights into student variables which influence student attitude towards the subject of Economics and the student's test score.

At the time of testing, the average student age in Year 11 was 16.4 years, and this ranged from 14.8 - 20 years. The average student age in Year 12 was 17.4 years, and this ranged from 16-23 years. There were close to equal numbers of males and females in both year levels.

Students reported better last year results in English than Mathematics, having received mainly 'B's in English and 'C's in Mathematics in 1997. Although both parents had predominantly completed secondary schooling, followed by university, 86% of students from each year level expected to obtain a university education.

Spending patterns of students revealed some surprising results, particularly when those who actually did spend on the items in question were examined. Most Year 11 and 12 students tended to spend money on food. Only 19% of Year 11 students and 32% of Year 12 students spent money on technical gadgets. However, those students who did spend money on them, spent rather a lot. Hence, four percent of Year 11s and eight percent of year 12s spend about $400 per month on technical gadgets. Overall, Year 11s spent, in total, an average of $245, while Year 12s spent $375 per month.

Both groups tended to spend over $50 each month on each of the following items: clothing, food and savings. Less than $20 per month was spent by Year 11s on videos and cosmetics, and by year 12s on videos only. The older students were more likely to spend significant amounts of money on cosmetics, food, technical gadgets. These results revealed that students appeared to be regularly spending reasonably large amounts of money, and had some experience in financial matters. Hence, topics in the study of Economics had the potential to be of great perceived relevance when taught in ways that are meaningful to students, eg interest rates and students' personal savings account, markets and trends in student buyer behaviour.

While the majority of students reported not spending any time on a number of the activities described below, descriptive summaries of the students who did partake in these activities sometimes revealed startling figures. For example, one hour was the average time spent playing PC games by students involved in this activity.

Between 50-60% of Year 11 and 12 students did not work in a paid job, received extra tuition, nor read in the school library. However, in excess of 20% of Year 12s were employed for more than nine hours of paid work each week while the overall mean for the employed students was 10 hours of paid work per week. Further analysis revealed that the more year 12s worked, the less homework they reported doing each night.

Students who read more in the school library tended to also receive higher amounts of tuition. A greater number of hours of tuition were associated with Year 11s who were employed in longer hours of paid work done. Approximately 20% of Year 11s and 12s received up to two hours of tuition each week, and 40 % had up to two hours of weekly home chores. The average amount of time spent by those doing these activities was approximately 2.5 hours weekly tuition, and four hours of home chores.

Of those who watched TV, Year 11s tended to watch more. The figures were very similar in the length of time spent reading, playing PC games, playing sports, doing all homework and also doing Economics homework. Of those who do chores at home, read in the school library and/or who are employed in paid work, Year 12s spent more time than Year 11 students on these activities.

The average time spent on Economics homework was 15 minutes for both year groups, although a large percentage of students spent up to 30 minutes. Those who reported doing so, tended to spend approximately 1.5 hours each night on all homework. Year 12s who reported spending greater time on all homework tended to have higher educational expectations.

In summary, paid work, nightly homework and home chores appeared to take up the majority of time of both year groups. Leisure type activities appear to consume significantly less time than these other responsibilities.

Approximately 50% of students when asked why they had chosen to study Economics, agave preferred reasons as:

(1) future career aspirations,

(2) fits the timetable, and

(3) their liking of the subject.

Year 12s preferred very much instructional methods incorporating group discussion, and they liked small group work. Year 11 students liked instructional methods incorporating small group work and group discussion. All other instructional methods were perceived as 'okay'.

Year 11s and 12s very much agreed that Economics lessons had helped in answering the test of Economic Literacy, and agreed very little that other lessons from Years 1-10 had contributed to their score. These findings imply that other school lessons tend not to relate and/or connect associated aspects of content with economics concepts, and that students may have difficulties detecting the relevance of other types of mediums, such as the newspaper and TV, in learning about Economics.

The following section will provide details on the achievement levels of student in Economics.

Students were given up to 40 minutes to answer a multiple-choice test on economic literacy. Test items covered the main content areas of Fundamental Concepts, Microeconomics, Macroeconomics and International Economics. With few exceptions, students needed less than the maximum amount of time available to complete the test. Hence, the test was not a speeded but a power test.

The Year 11 test comprised 42 items of which 13 were assigned to the Fundamental subscale, 12 to the Microeconomics subscale and 14 to the Macroeconomic subscale.

Figure 1 shows box plots for the performance of all Year 11 students in Queensland, all female and male students and the three different school types. Scores are plotted for the total scale as well as for the subscales of Fundamental Concepts, Microeconomics and Macroeconomics.

Figure 1 illustrates that across Queensland, students achieve the highest level of competence in Macroeconomics which covers concepts such as gross national product, inflation and deflation, monetary and fiscal policy. In contrast, students achieve the lowest level of competence in Microeconomics which covers concepts such as markets and prices, supply and demand as well as competition and market structure.

When examining the spread of scores, results show that the range of performance levels are greater for the Macroeonomics subscale than the Microeconomic subscale.

Significant differences in performance levels emerged between male and female students and between students attending different school types. Thus, male students outperform female students in the areas of Fundamental Concepts and Independent schools achieve at a higher level than students from State and Catholic schools.

The Year 12 test comprised 52 items of which 14 were assigned to the Fundamental subscale, 15 to the Microeconomics subscale and 12 to the Macroeconomic subscale and 11 to the subscale measuring International Economics. The latter scale was only added at Year 12 level as a teacher rating of the test items in the Central Queensland Pilot Study in 1997 had shown that the content required to answer these items is not taught until Year 12.

Figure 2 shows that Year 12 students achieve the highest level of competence in Microeconomics. This is in contrast to the findings for Year 11 students who showed the lowest performance on that subscale. Indeed, the Year 12 data illustrates clearly that students achieve significantly lower in the content areas of Fundamental Concepts, Macroeconomics and International Economics than in Microeconomics.

While the scores of male students were higher for the total as well as all four subscales, the only score for which the gender difference was significant was for the International Economics subscale.

A finding at the Year 11 level which was confirmed in the data for Year 12 was that students from Independent schools perform, on average, significantly better than students from State and Catholic schools.

In order to develop a hierarchical model of factors influencing Economic Literacy, correlations were calculated between variables on which information was collected in the student, teacher and school questionnaires.

At the student level, the a number of variables showed sizeable correlations with achievement. Thus higher educational expectations (SEXPED) were associated with higher performance. Age (AGE) operated differently between the year groups. The older Year 11 students tended to obtain lower scores in the test of Economic Literacy. These lower performing students may drop out from school before or during Year 12, or may repeat Year 11, and hence similar findings were not observed in the older sample.

Students enrolled in science tended to obtain higher scores (SSCIENCE). This is probably because sciences are taken at this year level by the higher achieving students with university aspirations. Those students who scored high grades in English (SLSTENG) and Mathematics last year (SLSTMATH) tended to be among the higher scoring students in the test of Economics Literacy. Year 11s who completed more homework obtained higher scores.

Longer hours spent on various activities tended to negatively impact on student scores. For Year 11 students, these activities included paid work (SPAIDJOB) and watching TV (SWATCHTV). In Year 12, longer hours doing home chores (SJOBHOME) were significant.

Variables pertaining to teachers' backgrounds and instructional techniques which were substantially correlated with achievement included:

At the school level, significant correlations with achievement in Economics emerged for the following variables:

However, such bivariate correlations between background variables and achievement are of limited usefulness for two reasons. First, other factors, such as socio-economic status, may have mediating effects on relationship. Second, different numbers of cases at the student, teacher and school levels, mean that some data either have to be aggregated (e.g. from the student to the school level) or disaggregated (e.g. from the teacher to the student level).

It has been shown that aggregation as well as disaggregation are likely to bias the estimates effect sizes in models of factors influencing school achievement (Sellin 1990). Hierarchical linear modeling, which is capable of incorporating data obtained at different levels simultaneously in one analysis, has been proposed as a possible solution (Raudenbush & Bryk 1986; Raudenbush & Willms 1991).

Based on the above correlational results, Figure 3 presents a possible hierarchical model which could be examined in future analyses.

One key element of the Economic Literacy Survey was the attempt to explore new ways of school assessment and data collection introducing the possibilities of PC- and WWW-based test administration.

The PC and WWW tests had been designed with the intention to preserve an utmost of identity in the format and 'handling'. Both new options, for example, allowed students during the test administration to scroll forward and backward at their own pace and liking - much the same as if they turn a page forward or backward. Corrections to answers made were also possible at any time. Basically, only the instructions to students how to proceed with testing were different. Regardless of the test option students were supervised by their Economics teacher.

It was of particular concern to find out if the three subgroups performed equally well or if the format of testing did, perhaps, influence the outcome. Table 3 and Table 4 illustrate how the three subgroups scored in the overall test as well as in the three subdomains.

When asked to choose the PC or WWW for testing several schools stated technical and administrative difficulties since often PC installations at school were booked out at the time of testing or insufficient numbers of (working) PCs were available. Economics teachers, who were meant to administer the test at their school site, were hesitant to try the PC option since they feared hard- or software related difficulties. Much in contrast, schools which opted to conduct the survey on the internet reported, in general, no major problems or technical obstacles.

While, on the surface, the scores of the students taking the test using paper & pencil seemed to trail behind those of the other subsamples this may be partly the result of unbalanced sample sizes. It is, however, encouraging to learn that, in general, no significant differences turned up between the three formats of assessing economic literacy among Year 12 students (Table 4). Similarly, no differences were found between the subgroups of paper & pencil and WWW at Year 11 (Table 3).

This findings advocate for a general shift to move towards PC- and internet-based testing where this is pedagogically appropriate and technically feasible.

It is well documented that the performance of an economy is directly tied to the capacity of its workforce (OECD 1992). On an international comparative scale the Organisation for Economic Cooperation and Development (OECD) measures, on a regular basis, educational indicators of its 29 member states.

The Directorate's work in education seeks to support the objective of making lifelong learning a reality for all, adopted by OECD Education Ministers in 1996. Though the OECD publication 'Education at a Glance: Indicators and Education Policy Analysis' is still the annual flagship of OECD publications in education other important work published recently includes Literacy Skills for the Knowledge Economy (OECD 1997) or Redefining Tertiary Education (OECD 1998). Among the important policy areas currently under study are the processes of transition from initial education to working life, policies for early childhood education and care, adult learning and the financing of lifelong learning.

Major steps were taken in 1997 towards conducting the first of an ongoing series of OECD international assessments of student achievement for 15 year-olds in reading literacy, mathematics and science. Twenty-five OECD countries have expressed a commitment to participate in the first round of the assessments, to be held in the year 2000. Similarly, the current ground work on economic literacy is meant to pave the way for an international study assessing literacy levels of senior secondary students in Economics.

The Economic Literacy Survey - Queensland 1998 showed that regular assessment on students' literacy levels in Economics is educationally desirable and logistically feasible. Hence, it is only a matter of developing an argument which convinces educational policy makers and governmental decision makers of the desirability to develop items that were used in the Queensland study into global indicators for the future.

The authors of this paper would like to thank all students, parents, teachers and school principals who made this study possible. We also thank Education Queensland, the Board of Senior Secondary School Studies, the Faculty of Business & Law (Central Queensland University), our research assistants as well as Apple Australia Pty and FileMaker Australia Pty for their administrative and logistical support.