A Comparison of Science Learning across Ten Countries Deidra J. Young and Barry J. Fraser Curtin University of Technology, Perth Paper presented to the 1993 Australian Association for Research in Education Conference, 22-25 November, Fremantle, Western Australia. A Comparison of Science Learning across Ten Countries Deidra J. Young and Barry J. Fraser Curtin University of Technology, Perth The International Association for the Evaluation of Educational Achievement (IEA) conducted a study of achievement in science in 19 countries in 1970 (First International Science Study) and in 24 countries/educational systems in 1983/84 (Second International Science Study). There were ten countries who participated in both of these studies. The IEA is a voluntary research organisation and does not select countries to participate in its studies. Rather, research centres in each country elect to participate as long as they have the experience and financial resources to conduct and fund the study. The participating countries plan the study on a cooperative basis, taking care to ensure that the test instruments are fair to all countries in terms of curriculum and culture. Additionally, care was taken that all student background questions, attitude scales and other measures were comparable cross-nationally. The Second International Science Study (SISS) sampled students from 24 countries and from three age groups: 10-year-old, 14- year-old and year 12 students (Rosier & Keeves, 1991; Postlethwaite & Wiley, 1992; Keeves, 1992). In this study, the 14-year-old student sample was analysed. The complex sample design meant that the normal assumptions of simple random sampling could not be made if statistical significance was to be tested in a valid manner. For this reason, a multilevel model was developed which accounted for the nested nature of the data. The range of variables available in the SISS from which selection could be made included school resources and environment, teacher and student characteristics and opinions. For the 14-year-old student population, there were more than 350 separate student variables to choose from. The magnitude of this database has provided educational researchers with a remarkable opportunity to examine science education in these countries. This research focused upon data from ten countries from the Second International Science Study, specifically the 14-year old students (Population 2) in Australia, England, Finland, Hungary, Italy, Japan, the Netherlands, Sweden, Thailand and the United States of America. In these countries, most 14- year-old students were required to be in school and science was generally compulsory (with the exception of Thailand). The purpose of this research was to investigate the relationship between the student's reported description of science learning and their performance in a science achievement test. This relationship was analysed using multilevel modelling for each of the ten countries. Classroom Environment Research International research into the conceptualisation, measurement and investigation of perceptions of psychosocial characteristics of the learning environment of classrooms at the primary, secondary and higher education levels have used classroom environment instruments both as predictor and criterion variables in a variety of research studies (Chavez, 1984; Fraser, 1986, 1989; Fraser, in press; Fraser & Walberg, 1991; MacAuley, 1990). The use of student perceptions of classroom environment as predictor variables in several different countries has established consistent relationships between the nature of the classroom environment and various student cognitive and affective outcomes (Fraser, 1986; Haertel, Walberg & Haertel, 1981). In addition, research involving a person-environment fit perspective has shown that students achieve better where there is greater congruence between the actual classroom environment and that preferred by students (Fraser & Fisher, 1983). In this analysis of the Second International Science Study, the students' reported perceptions of their science learning environments were investigated as predictors of their science achievement. The Sample and Target Populations While there were 23 countries participating in the Second International Science Study, the purpose of these analyses was to examine data from a selection of ten countries only. These ten countries were selected in order to maintain comparability with previous studies (Keeves, 1992; Kotte, 1992). While this paper focuses on Population 2, there were three age groups targeted in the Second International Science Study. Population 1 was the 10-year-old students. This was the upper primary school level in most countries, where students were taught by a general class teacher. The definition of Population 1 included students at the year level (grade level) where most of the students were 10 years of age at the time of the testing programme. Population 2 was the 14-year-old students. This was the lower secondary school level in most countries, and in many countries was the last point in the school system where 100 percent of an age group is still in compulsory schooling. The definition of Population 2 included students at the year level (grade level) where most of the students were 14 years of age at the time of testing. Population 3 consisted of students in the terminal or pre- university year of full-time secondary education programmes commonly designed to prepare students for tertiary studies. There were two subgroups to this population: Population 3S comprised students studying science at a level which would enable them to proceed to tertiary studies in science and Population 3N comprised students not studying science at a level which would enable them to proceed to tertiary studies in science. For the purposes of this study, Population 2 was selected for analysis. This was because the sample of students were more likely to represent their age group, with compulsory schooling being prevalent in these countries. The maximum age for compulsory schooling in the ten countries under investigation ranged from the age of 13 in Thailand to the age of 17 in the Netherlands. Also, the Population 2 sample represented 98 to 99 percent of the age cohort in school for all ten countries, except Thailand which was 32 percent of the age cohort in school. Clearly in this study, Thailand had larger differences in their educational system, when compared with the other nine countries. For Hungary, Italy and Thailand, there was a marked increase in retention of students at school since the First International Science Study in 1969. Science Learning Environment The students' perceptions of the science learning environment are most likely to influence their learning, irrespective of the actual facilities provided and the teacher's strategies used to teach science. Keeves and Dryden (1992, pp. 187-207) described teaching and learning in science classrooms from three general perspectives. First, there is the perspective of teaching by imparting information (teacher-directed learning or transmission of knowledge). Secondly, there is the perspective of teaching as meeting the needs of the students (student participation). Thirdly, there is the scientific perspective in which the learning of science is seen as a process of investigation (practical work and open- ended inquiry learning). These three views of science learning were termed instruction, participation and investigation and reflect a learning environment which is passive, sharing and active, respectively. In this study, the three descriptive scales associated with views of science teaching and learning were defined as follows: 1. Student Participation: The student reports being able to make a choice of science topics to be studied, doing field work outside the classroom, being permitted to make up problems and working out methods and solutions to problems. In addition, the teacher uses the students' ideas and suggestions in planning science lessons. 2. Teacher Directed Learning: The student reports that the science teacher starts lessons with an explanation of work to be covered and a reminder of what was taught in previous lessons, finishes with a summary, explains the relevance of the work taught, conducts demonstration experiments and helps students with difficulties in learning science. 3. Practical Work: The student reports doing practical work in small groups during science lessons, with written instructions or with instructions given by the teacher. Reports of practical work are written up for homework. Students were asked to give their views by responding to statements, as described in Table 1, and by indicating whether they considered that the activity involved in each statement often takes place, sometimes takes place or never takes place (coded 1, 0.5 and 0 respectively). The three descriptive scales were calculated by taking the mean of the items, so that the scales ranged in value from 0 to 1. If more than 20% of the items were missing from a scale, then the scale was set to missing for that student. Science Achievement The science achievement test for 14-year-old students consisted of 30 common core science test items and 40 rotated test items (4 tests were available), with students being required to take two of the rotated tests (20 items). Therefore, the maximum possible score was 50, although there were 70 possible science test items available. These science test items were multiple choice only, although they did cover a range of cognitive abilities and science content areas. The estimated mean science achievement score for each country is presented in Table 2, along with the standard errors of measurement, intra-class correlations and sample sizes by school and student. These are discussed further in a later section. Methodology Because these datasets comprised students nested within schools, there was bound to be a certain amount of variability between schools. While schools may form part of the same educational system within a country, they may have entirely different cultures, curricula and organisational environments. In addition, students within a classroom may experience different learning environments depending upon the teacher's style, characteristics and the characteristics of the other students in the classroom. There are many factors which can influence student performance, both at home and at school as well as the student's own internal influences. In this study, we chose to examine the students' perceptions of their own learning environment in science classes and how these perceptions related to their achievement in science. In order to separate the school level differences from the student differences in science achievement, we used a methodology usually referred to as multilevel modelling or hierarchical linear modelling (HLM). The use of powerful computers to analyse large databases during the First International Mathematics Study focused on the home and school variables influencing student achievement, while using the student as the unit of analysis (Husˇn, 1967). However, this type of analysis did not adequately address the variability of schools contributing to statistical tests of significance. Studies in the United States tried to compensate for this problem by only looking at the school differences; for example, Coleman et al. (1966) used the school as the unit of analysis. Further, a large database was analysed by Peaker (1967) in England by examination of between- school means (aggregated student data) and pooling between students. However, these studies ignored the differences between students within-schools which can contribute towards explaining the variance. The controversy over the most appropriate unit of analysis has continued until the importance of a different approach to educational research was first proposed by Cronbach: The majority of studies of educational effects - whether classroom experiments, or evaluations of programs, or surveys - have collected and analysed data in ways that conceal more than they reveal. The established methods have generated false conclusions in many studies. (Cronbach, 1976, p. 1) Most educational research revolves around students who receive schooling in classrooms located within schools, within school districts, within states, etc. The grouping of students, classes and schools occurs in a hierarchical order with each group influencing the members of the group in thought and behaviour. The nature of these hierarchical structures produces multilevel data. Theories about the effects of the multilevel structure of education (the different levels of the educational hierarchy) should lead to attempts to specify models which involve the analysis of multilevel educational data. Burstein (1980) believes that these theories eventually will replace experimental design and analysis with the natural design and analyses that evolve from the multilevel structure of data. The amount of variation in estimates of variables affecting academic achievement across different levels of analysis cannot be ignored by serious educational researchers. In particular, the socioeconomic status of the student and of the school have been shown to consistently account for a large amount of variation in achievement both between students and between schools. Traditional linear models on which most researchers rely require the assumption that errors are independent, yet most subjects are 'nested' within classrooms, schools, districts, states and countries so that responses within groups are group dependent. To ignore the nested structure of this type of data ultimately will give rise to problems of aggregation bias (within-group homogeneity) and imprecision (Raudenbush, 1988). The Hierarchical Linear Model (HLM) provides an integrated strategy for handling problems such as aggregation bias in standard error estimates and erroneous probability values in hypothesis testing of school effects. For this study, HLM was chosen as the model most appropriate to study school and student effects relating to science achievement, and HLM (Bryk, Raudenbush, Seltzer & Congdon, 1989) was selected as the computer package most suited to analyse the large amount of data in SISS. The use of the HLM in order to investigate the influence of the organisational structure of the school on student performance has been documented by Bryk and Raudenbush (1989, pp. 159-204), Lee and Bryk (1989) and Raudenbush and Bryk (1986). The present study sought to examine the role of school effects in explaining student differences in science achievement. Research on school effects has been conducted with a set of data analysed at the individual student level, with the assumption that classrooms and schools affect students equally. However, when the effects vary among individuals and their contexts, this type of statistical analysis can be misleading (Bryk & Raudenbush, 1987). Ordinary least squares analysis provides information about the total variance, but can only break this total variance into the between- and within-school effects. The between-school effect may be influenced by school level variables, such as the affluence of the school. Research which endeavours to explain variations in student outcomes by first decomposing observed relationships into between- and within-school components have used the hierarchical linear model to examine the effects of the home and the school on student achievement (Young, 1991a, 1991b, in press; Young & Fraser, 1993, in press), but this study focuses on the effect of the science learning environment. Results And Discussion The partitioning of variance in science achievement among students into the within- and between-school components was achieved using the HLM computer package (Version 3.01, Scientific Software, April 1992; Bryk, Raudenbush, Seltzer & Congdon, 1989). A random mean science achievement estimate was specified for the within-school model: Scienceij = b0j + Rij Equation 1 where i = 1, . . ., nj students in school j, j = 1, . . ., J schools, Scienceij represents science achievement of student i in school j, b0j represents the mean science achievement for students in school j and Rij represents random error of student i in school j. At the school level, the school mean science achievement is a function of the grand mean, g00, with random error, m0j : b0j = g00,+ m0j Equation 2 The grand mean in these analyses were estimated, along with their standard errors, and presented in Table 2 for all ten countries. In these analyses of the random model for Australia, the variance in science achievement was found to be 16 percent at the school level (o(^,t)00 = var(m0j) » 10.13), while 84 percent of the variance was related to student level differences (o(^,s)2 = var(rij) » 52.56); these estimates indicate that most of the variation in science achievement for Australia was at the student level, although a substantial proportion was between schools. Similar results were found for England, Hungary and Italy with 19, 26 and 25 percent of the unexplained variance at the school level. However, there were some differences noted for other countries. Finland, Japan and Sweden had very low intra-class correlations, that is there were few school differences in mean science achievement. This could either be a reflection of their educational systems or of the sample selected for the SISS study. That is, their schools could be very similar or the sample of schools selected could be very similar. On the other hand, the Netherlands, Thailand and the United States had much higher intra-class correlations (0.52, 0.29 and 0.34 respectively). That is, their schools varied significantly in average student achievement in science. Again, this could be a reflection of a much better, more varied sample of schools selected for this study, or simply that these countries have a more polarised educational system. It is important that these differences between schools are kept in perspective when comparing countries. For the purposes of this study, the three science learning environment scales were modelled on science achievement at both the student and school levels as the following model explains: Scienceij = b0j + b1j(Stdirectij - o(-,X)stdirect.) + b2j(Tstructij - o(-,X)tstruct..) + b3j(Pracworkij - o(-,X)pracwork.) + rij b0j = g00 + g01Stdirectj + g02Tstructj + g03Pracwork + m0j b10 = g10 b20 = g20 b30 = g30 In the above model, each of the three science learning scales were centred around the grand mean, so that the intercept, b0j, represents the grand mean science achievement when adjusted for these science learning scales. Only the intercept was allowed to vary across schools, while the other science learning slopes were kept fixed. That is, their variability between schools was constrained to zero. The b coefficients represent student level slopes which can then be the outcomes for school level effects g. In these analyses, only the intercept was modelled as an outcome. The results of these analyses are provided in Table 3 for the ten countries at the 14-year-old level. Those coefficients which were statistically significant are marked with an asterisk. The school level predictors for the intercept b00 were the school average for student participation, g01, teacher directed learning, g02, and practical work, g03. The mean science achievement was the intercept g00. The student level predictors were their reported perceptions of their science learning environment, that is, student participation, b10, teacher directed learning, b20, and practical work, b30. The error term for the student level model was rij, while the error term for the school level model was m0j, with their variances given in Table 3. There are a few consistent patterns seen in Table 3 across the ten countries. Firstly, the significance of the intercepts indicates that these science learning scales do not fully explain the variance in science achievement. There are likely to be other factors which would better explain this variance and these will be discussed in another place (Young & Fraser, 1994). While the average student participation and teacher directed learning scales were generally not significant, they were almost always negative. Similarly, student directed learning slopes at the student level were strong and negative for all countries analysed. This tends to indicate that the more students participated in their science learning management, the lower their science achievement became. The average teacher directed learning was not significant, however student level analysis revealed a positive slope for Hungary, Italy, Japan, the Netherlands and Sweden. For these countries, it appeared that students' performance in science improved when teachers managed their learning in a direct manner. For all other countries, however, teacher directed learning did not impact one way or another on student science achievement. The most significant finding in these analyses were the consistently large and positive effects of the practical work scale on science achievement. It appeared that schools with a larger reported practical work component to their science lessons had a much improved science achievement. In addition, students who reported being more involved in practical work also had higher science achievement score. The importance of this unexpected and strong practical work effect on science achievement has implications for both the school and the science teacher. The explanation of variance at both Level-2 (school) and Level- 1 (student) varied from country to country. Most of the reduction in unexplained variance was at the school level (between schools), ranging from 7 percent for Hungary to 43.4 percent for the Netherlands. While most of the unexplained variance was at the student level (within schools), very little of this variance was reduced. Of course this was to be expected, as most research points to the so called 'unalterables' accounting for student differences e.g., socioeconomic background, prior learning experiences, potential ability, gender and ethnicity. Summary This paper attempted to compare the effects of the student reported science learning environment across ten countries. While these countries are not necessarily comparable in terms of their educational systems, it is worthy to note any consistency in patterns of significant effects on science achievement. The dramatic finding that the increased practical work component in science lessons was associated with improved science achievement by 14-year-old students across all ten countries examined in these analyses. While these preliminary findings examine the science learning environment scales on their own, further analyses which make adjustments for the student home backgrounds and abilities will certainly improve the model (see Young & Fraser, 1994). References Bryk, A.S., & Raudenbush, S.W. (1989). Toward a more appropriate conceptualisation of research on school effects: A three-level hierarchical linear model. In R. D. Bock (Ed.), Multilevel analysis of educational data. San Diego: Academic Press. Bryk, A.S., Raudenbush, S.W., Seltzer, M., & Congdon, R.T. (1989, 1988, 1986). An introduction to HLM: Computer program and users' guide. Chicago: University of Chicago. (Available from Scientific Software, Chicago). Burstein, L. (1980). The analysis of multilevel data in educational research and evaluation. Review of Research in Education, 8, 158-233. Chavez, R.C. (1984). The use of high inference measures to study classroom climates: A review. Review of Educational Research, 54, 237-261. Coleman, J. S., Campbell, E. Q., Hobson, C.J., McPartland, J., Mood, A.M., Weinfeld, F.D., & York, R.L. (1966). Equality of educational opportunity. Washington, D.C.: U.S. Government Printing Office. Cronbach, L.J. (1976). Research on classrooms and schools: Formulation of questions, design, and analysis (Occasional Paper). Stanford, California: Stanford Evaluation Consortium. Fraser, B.J. (1986). Classroom Environment. London: Croom Helm. Fraser, B.J. (1989). Twenty years on classroom environment work: Progress and prospect. Journal of Curriculum Studies, 21, 307-327. Fraser, B.J. (in press). Classroom and school climate. In D. Gabel (Ed.), Handbook of Research on Science Teaching and Learning. New York: Macmillan. Fraser, B.J., & Fisher, D.L. (1983). Student achievement as a function of person-environment fit: A regression surface analysis. British Journal of Educational Psychology, 53, 89- 99. Fraser, B.J., & Walberg, H.J. (Eds.) (1991). Educational Environments: Evaluation, Antecedents, and Consequences. Oxford: Pergamon Press. Haertel, G.D., Walberg, H.J., & Haertel, E.H. (1981). Socio- psychological environments and learning: A quantitative synthesis. British Educational Research Journal, 7, 27-36. Husˇn, T. (1967). International study of achievement in mathematics: A comparison of twelve countries. Stockholm: Almqvist and Wiksell. Keeves, J.P. (1992). The IEA study of science III: Changes in science education and achievement: 1970 to 1984. Oxford: Pergamon Press. Keeves, J. P., & Dryden, M. (1992). Learning of science at population 2: 1983-84. In J. P. Keeves (Ed.), The IEA study of science III: Changes in science education and achievement: 1970 to 1984. Oxford: Pergamon Press. Kotte, D. (1992). Gender differences in science achievement in ten countries. Bern: Verlag Peter Lang. Lee, V. E., & Bryk, A. S. (1989). A multilevel model of the social distribution of high school achievement. Sociology of Education, 62(2), 246. MacAuley, D.J. (1990). Classroom environment: A literature review. Educational Psychology, 10, 239-253. Peaker, G.F. (1967). The regression analyses of the national survey. In United Kingdom, Department of Education and Science, Children and their primary schools (Plowden, Chairman). London: HMSO. Volume 2, Appendix 4. Postlethwaite, T.N., & Wiley, D.E. (Eds.). (1992). The IEA study of science II: Science achievement in twenty-three countries. Oxford: Pergamon Press. Raudenbush, S.W. (1988). Educational applications of hierarchical linear models: A review. Journal of Educational Statistics, 13(2), 85-116. Raudenbush, S.W., & Bryk, A.S. (1986). A hierarchical model for studying school effects. Sociology of Education, 59(1), 1-17. Rosier, M.J., & Keeves, J.P. (Eds.). (1991). Science education and curricula on twenty-three countries. Oxford: Pergamon Press. Young, D.J. (1991a). Gender differences in science achievement: Secondary analysis of data from the Second International Science Study. Doctoral thesis, Science and Mathematics Education Centre, Curtin University of Technology, Perth. Young, D.J. (1991b). Multilevel analysis of sex and other factors influencing science achievement. In L. J. Rennie, L. H. Parker & G. M. Hildebrand (Eds.), Action for Equity: The Second Decade. Contributions to the sixth international GASAT conference. Volume one: Schooling (pp. 383-391). Perth: National Key Centre for School Science and Mathematics. Young, D.J. (in press). Single-sex schools and physics achievement: are girls really advantaged? International Science Education Journal. Young, D.J., & Fraser, B.J. (1993). School effectiveness and science achievement: Are there any sex differences? Journal for School Effectiveness and School Improvement. 4(4). Young, D.J., & Fraser, B.J. (In Press). A multilevel model of sex differences in science achievement: The Australian Second International Science Study. Journal of Research in Science Teaching. Young, D.J., & Fraser, B.J. (1994, January). A cross country comparison of factors contributing to gender and socioeconomic equity. Paper presented at the International Congress for School Effectiveness and Improvement. The World Congress Centre, Melbourne, Victoria. ˙ Table 1. The Science Descriptive Scales and Items for Population 2 Student Participation (Stdirect) 1. We are allowed to make our own choice of scienc topics to study. 2. The teacher uses our ideas and suggestions when planning science lessons. 3. We do field work outside the classroom as part of our science lessons. 4. In our practical work we make up our own problems and then the teacher helps us to plan experiments to solve them 5. When we do experiments the teacher gives us problems to solve and then leaves us to work out our own methods and solutions. 6. In our practical work, we make up our own problems and work out our own methods to investigate the problems. Teacher-Directed Learning (Tstruct) 1. At the start of each science lesson, the teacher reminds us about the work we covered during previous lessons. 2. At the start of each science lesson, the teacher explains the work we have to cover during the lesson. 3. At the end of each science lesson, the teacher gives a summary of what was taught in the lesson. 4. The teacher does demonstrations to help explain scientific ideas. 5. The teacher explains how the science we do is relevant to our own lives. 6. The science teacher helps students who have difficulties with learning science. Practical Work (Pracwork) 1. For science homework, we write up reports of our laboratory and practical work. 2. We do practical work (experiments) as part of our science lessons. 3. The science class breaks into small groups of students to do practical work (experiments). 4. When we do experiments, the teacher gives us instruction about what to do. 5. When we do an experiment, we use a book or other written instructions to show us how to do it. ˙Table 2. Estimated Mean Science Achievement and One-way ANOVA for Ten Countries and 14-Year-Old Students, 1983/84 Fixed Effect Coefficient AustraliaEngland Finland Hungary Italy Japan The SwedenThailand USA Netherlands Grand mean achievement, b00 Intercept 30.06 27.55 30.14 35.19 26.03 33.42 30.41 34.11 28.03 30.98 Standard Error .23 .31 .21 .41 .27 .15 .41 .35 .36 .52 Random Effect Variance Components - One Way ANOVA for Science Achievement Level-2 effect, m0j 10.13 11.86 2.45 14.85 14.13 2.44 35.35 6.51 11.43 21.62 Level-1 effect, rij 52.56 51.28 45.20 42.82 42.80 62.23 32.20 88.87 28.29 41.61 Total Variance (L1 & L2) 62.69 63.14 47.65 57.67 56.93 64.67 67.55 95.38 39.72 63.23 Intra-Class Correlation0.16 0.19 0.05 0.26 0.25 0.04 0.52 0.07 0.29 0.34 Sample Size Schools 233 145 89 98 222 200 224 137 96 88 Students 4917 3118 2546 2515 4622 7611 5025 1263 3780 1958 D:\hlm\1993\siss\country\country01.out ˙ Table 3. Estimated Effects of Science Learning Environment on Science Achievement for 10 Countries and 14-Year-Old Students, 1983/84 Fixed Effect Coefficient AustraliaEngland Finland Hungary Italy Japan The SwedenThailand USA Netherlands Grand mean achievement, b00 Intercept, g00 28.30** 21.51** 28.54** 25.97** 28.40**29.85** 29.94** 30.94** 25.31** 29.25** Student Participation, g01 -5.83 -10.15* -.01 -10.26 -15.58** 1.89 -41.71** 6.73- 18.50**-6.51 Teacher Directed Learning, g02-4.71 -4.43 -6.54 8.48 1.81 1.15 .08 -7.63 -5.61 -3.80 Practical Work, g03 8.15** 14.86** 8.63** 10.97 9.87** 3.52* 14.78** 7.17 18.57** 8.62** Student Directed Learning, b10 -10.00**-15.51** -6.22** -7.93**-1.52** - 5.59** -6.84**-10.23**-5.25**-7.78** Teacher Directed Learning, b20 -.64 1.65* -.32 1.85* 1.61** 4.75** 1.40** 5.70** -.31 .17 Practical Work, b30 7.01** 4.74** 1.85** 2.96** -1.11* .92* .73* 11.37** 3.63**3.85** Random Effect Variance Components - One Way ANOVA for Science Achievement Level-2 effect, m0j 10.13 11.86 2.45 14.85 14.13 2.44 35.35 6.51 11.43 21.62 Level-1 effect, rij 52.56 51.28 45.20 42.82 42.80 62.23 32.20 88.87 28.29 41.61 Total Variance (L1 & L2) 62.69 63.14 47.65 57.67 56.93 64.67 67.55 95.38 39.72 63.23 Random Effect Variance Components With Student and School Level Science Learning Environment Scales Level-2 effect, m0j 7.54 8.30 2.10 13.81 12.69 1.97 20.00 5.77 7.75 14.78 Level-1 effect, rij 49.62 45.88 44.48 41.52 42.60 61.51 31.61 81.49 27.52 40.32 Percentage Reduction Level-2 25.6 30.0 14.3 7.0 10.2 19.3 43.4 11.4 32.2 31.6 Level-1 5.6 10.5 1.6 3.0 .5 1.2 1.8 8.3 2.7 3.1 Reliability of OLS Regression-Coefficients Estimates Mean achievement 0.757 .784 .564 .889 .854 .518 .931 .371 .914 .877 ** Statistically significant at p<.01 * Statistically significant at p<.05 D:\hlm\1993\siss\country\country08.out