School Level Variables as Predictors of Individual Student Achievement

Sid Bourke

University of Newcastle

______________________________________________

Abstract

It is generally recognised that the home background of students is important for their achievement and progress through school, however, obtaining accurate information is becoming increasingly difficult. Concerns about privacy of family structure, income and occupation with blended families and whether parents are working, and the difficulty of obtaining accurate data from students means that parents have to be approached for the information as well as approval. Low response rates from parents questioned about such matters are of concern.

School-level indicators of socioeconomic status, although coarse, may consist of more than the sum of family backgrounds of students attending the school. They add a "community" context, especially in the case of state primary schools which serve a defined local area. The usefulness of school-level data on poverty and ethnicity in predicting individual achievement of students in the Advanced Program was tested for a school district in South Carolina, USA. Although other school and individual indicators were considered, the major focus of this paper is the relationship of poverty of the total school, as measured by the proportion of students receiving free or reduced-price lunches, with the MAT7 basic skills reading and mathematics scores of 1394 students in Years 3 to 5.

_________________________________________________________

Paper presented at the Annual Conference of the Australian Association for Research in Education, Adelaide, Nov/Dec 1998.

The author may be contacted as follows:

Phone: 02 4921 5901, Fax: 02 4921 6992, Email: edsfb@cc.newcastle.edu.au

SCHOOL LEVEL VARIABLES AS PREDICTORS OF

INDIVIDUAL STUDENT ACHIEVEMENT

OBTAINING HOME BACKGROUND INFORMATION

It has been generally recognised for some time that the home background of students is important for their achievement and progress through school Marjoribanks, 1974 p.99; Fuligni & Stevenson, 1994 p.2624; Keeves, 1994 pp.3870-3871). In particular the source of home background measures has tended to be on one or both of the education level or the occupation of one or both parents. However, it has always been problematic obtaining a range of accurate home background such as parent education, occupation, income and other socioeconomic and socio-educational information from students, especially younger students in primary schools. The first problem is whether the younger students actually know the information being requested, or whether it must be obtained directly from parents and guardians. The second problem is related to a changing policy concerning the collection of data from students. It is increasingly the case that explicit approval from parents and guardians is required before students can be asked to provide such information.

For both of these reasons it is becoming increasingly difficult, especially for researchers not working within state departments of education or other education systems, to obtain information about the home backgrounds of students. As suggested above, to obtain accurate home and family information it may be necessary to request it from parents but, in any case, concerns about privacy of family structure, income and occupation with blended families and when parents are not working, more often mean that it is necessary to ask parents and guardians to provide such information. Even if approval is gained from state education department and university ethics committees to obtain this and other information directly from students, it is now common practice that the prior return of parental approval notes to schools is required, rather than a reliance on the assumption that no response was a tacit indication of approval, or at least of lack of disapproval which previously was often taken as the same thing.

This change in practice has exacerbated another chronic problem for university-based educational research conducted with the cooperation of schools. Notoriously low return rates of forms from parents simply asked to sign approvals tend to be even lower when parents are also asked to respond to questions about their occupation, income and other family and home practices. Such a threat to response rates requires either increasing personal contact with parents, thus increasing the time and expense of obtaining home background information, or finding other ways to obtain comparable information. An antidote is needed to a potential reduction in effective sample size and possible loss of representativeness for studies caught up in the reality of high non-return rates of home background information.

SCHOOL-LEVEL AND INDIVIDUAL-LEVEL INDICATORS

In part because of the above problems, there is considerable interest by educational researchers in school-level variables as indicators of socioeconomic status and other home and social measures. School-level indicators of the socioeconomic status of its clientele have two particular characteristics: accurate school indicators are much more accessible to researchers, but such indicators often consist of more than the sum of the family backgrounds of students attending the school. The latter characteristic may be a mixed blessing for researchers.

Many school-level indicators add a "community" context, especially in the case of state primary schools which tend to serve a small, defined local area. The same is generally true for state secondary schools, except that the geographic area is extended and greater variability is present in the school population. The definition of "community" for private schools depends less on geography, but other social factors may well make their sense of community stronger than is generally the case for state schools.

At best, however, school-level indicators of home background are generally considered no more than a coarse guide to individual student information, especially when individual student outcomes such as achievement or attitudes are the outcomes of interest. Obviously students at any one school have different home backgrounds, so any school measure has a range of error at the individual student level, not shared by similar individual student measures. The common wisdom is that if student measures are available, they are to be preferred to comparable school measures when student outcomes are the focus of a study.

Student socioeconomic status

Some Australian information on the extent to which this clearly reasonable assumption is true in particular circumstances is available. For example, in a methodological study of the relationships of secondary student achievement in literacy and numeracy with individual socioeconomic data and a range of similar area-based aggregated indices, Ainley et al (1995), found that similarly-structured aggregated data reduced the correlations between achievement and socioeconomic status by moderate amounts when compared with individual student data. A comparison was made between aggregated coefficients of the order of 0.21 to 0.23, with the individual coefficients being approximately 0.30 (p.73). Although there is an obvious diminution in the strength of the relationship when working with aggregated data, it could be argued that the loss is not such as to deny the usefulness of the aggregated measures.

Secondary schools normally have a more heterogeneous student body with respect to socioeconomic status than primary schools, so one might expect that smaller differences would exist for primary students in the relative strengths of relationships of achievement with individual and with aggregated socioeconomic data. In any case, at the primary level, the correlation between individual socioeconomic background and student achievement has been found to be lower than that for the secondary level, and of the order of 0.24 (Ainley et al, 1990 p.103).

We now turn briefly to studies using solely individual student data in an attempt to gauge the likely importance of socioeconomic status variables for student achievement in the Australian context. Williams et al (1980 pp.59-60) found consistent and substantial effects of a socioeconomic index made up of parental education and occupation on primary student literacy and numeracy achievement. Using a socioeconomic index based on an occupational prestige scale, Ainley et al (1990 pp.82-83) found similar relationships between socioeconomic status and achievement in mathematics and reading, with correlation coefficients being in the range of 0.22 to 0.28.

Student ethnicity

Ethnicity in Australian schools has been found to have inconsistent relationships with basic skills achievement. In one major study of primary schooling in Victoria, it was found that reading comprehension was related to ethnic background, with students who had both parents from non-English-speaking backgrounds, or who themselves were born in a non-English speaking country, having lower achievement (Ainley et al, 1990 pp.88-89). Mathematics achievement did not differ at Year 5 level, although it did at Year 6. It was suggested that this inconsistency may have been a result of the Year 6 being more language dependent than the Year 5 test. For secondary students, ethnicity could generally account a little more consistently for variance in achievement, perhaps of the order of approximately three per cent (Clifton, 1994 p.4893).

Earlier studies of primary achievement across Australia had also suggested that the effects of ethnicity on literacy and numeracy achievement were variable (Bourke, 1977 p.93), and small and statistically insignificant (Williams et al, 1980 p.58). The study reported by Bourke indicated that the inconsistency was due to the effects on achievement varied with the region of origin of migrant families.

Further work on the use of school-level variables in predicting individual student achievement is currently being undertaken (for example, Highett, 1998). The levels of relationships between school-level variables of ethnicity and socioeconomic background and primary-age student achievement are tested through the use of multiple regression analyses in the study reported in this paper.

THE PRESENT STUDY

The data available consist of ability, achievement and background measures for 1394 students in Grades 3 to 5, and school-level data on the 30 elementary (primary) schools they attend in an urban school district in South Carolina, USA. The students are in the Advanced Academic Program (AAP), and consequently are among the most able students in the District. Data provided indicates that 16.6 per cent of students in the District are in the AAP, with individual schools ranging from a low of 4 per cent to a high of 33 per cent of their students in the AAP.

There are two, or perhaps two-and-a-half tiers in the selection of students into the AAP. First students may be selected at the State level because they are in the top decile on one of the MAT7 reading or mathematics tests or OLSAT scores. If not selected at the State level, students may be selected at the district level if they are in the next decile on at least one of the measures. Finally a few additional students are selected as a result of representations to the school principal. State or local selection for each student was recorded in the data set.

All African-American and Caucasian AAP students in the District for whom sufficient data were available (not including the OLSAT scores for which data were frequently missing) were included in the sample on which this study is based. Because of inter-school and inter-District student mobility and other reasons for incompleteness of the data available, the 1394 students included in these analyses represent only two-thirds of the total of 2060 students recorded by the District as being in the Program in Grades 3 to 5. Mobility was a feature for approximately 16 per cent of all District students (Go, 1998), and thus probably accounted for half of the missing data.

Student Data

Personal Data. In addition to student age (in months) and gender, student ethnicity was recorded as African-American or Caucasian. For simplicity, the small number of 30 Asian/Pacific students (less than 1.5% of the total) was excluded.

Mean age was 117.5 months with students grouped into the three grade levels involved (see Table 1). The total sample consisted of 52 per cent female students and gender distribution across the grades is shown in Table 1. The proportion of students classified as Caucasian (overall 69% of the sample was African-American), and the proportions of students who were locally identified for the Program (66%), as compared with state identified (34%) are also shown.

Achievement Measures. The student measures of achievement were the percentile ranks on the MAT7 basic skills reading and mathematics tests, and total 3R scores. Because only the AAP students were included, the test distributions were strongly negatively skewed and consequently were normalised with the same mean and standard deviation as the original measure. The means and standard deviations for each measure are shown in Table 2.

Ability Measure. The Ravens test percentile rank was taken as the ability measure used for each student in the Program. This distribution was also highly negatively skewed and was also normalised (again see Table 2).

School Data

All 30 schools in the District were included in this study. School size ranged from 172 to 864, with a mean of 446 (sd=80). The larger schools tended to have higher proportions of their students undertaking the AAP (r=0.55, n=30). The school variables of major interest are mentioned briefly below and are shown in Table 3.

Poverty index. The proportion of students receiving free or reduced-price lunches was taken as a measure of poverty of the schools community. The range across the 30 schools was 31 to 100 per cent with a mean of 79 per cent.

Ethnic mix. The proportion of African-American students attending each school in this District ranged from 32 to 100 per cent, with a mean of 84 per cent. The use of this school aggregate measure can be compared with use of the dichotomous variable indicating individual student ethnicity which was also available.

Student Retention and Suspension. The rate of student retention in grade across the schools over one year ranged from zero to 13 per cent (mean=2.5%). The level of retention is, of course, a mix of school policy and individual student achievement. The rate of student suspensions, also over a school year, ranged from zero to 23 per cent (mean=6.8%), and may be taken as a measure of student morale and discipline policy in the school as a whole.

Relationships with school achievement

Initially simple correlational analyses of the student and school variables with individual student achievement were undertaken, followed by multilevel regression analyses using the MlwiN program (Goldstein et al, 1998) to determine the best sets of predictor variables for reading, mathematics and the combined 3R achievement tests. A mulitlevel analysis was used for two reasons. First, the analysis was appropriate because of the existence of variables at two levels of aggregation, student and school and, secondly, because significance testing required a recognition of the effect of student clustering within schools. Grade level was a third, intermediate, level in the analyses. The relative effects of student and school variables are considered in a series of three-level analyses with respect to their appropriate standard errors.

Correlations of student and school variables with individual student achievement

These relationships were examined first using simple correlations between the student and school variables described above and individual student achievement on the three MAT7 measures. A brief summary for each variable is now given and details are shown in Table 4.

Grade level. Surprisingly, the correlations of achievement with grade level were low, particularly for reading and 3R tests. Even more surprisingly, the correlation of grade level with mathematics score was negative.

Gender. Female students generally had higher scores in reading (r=0.159) and the 3R test (0.179), but not in mathematics, where there was no gender difference in achievement.

Ethnicity. Caucasian students generally had higher reading scores (r=242) and, to a lesser extent, higher 3R scores (0.210) than African-American students, but ethnicity was not related to mathematics score.

Identification. As would be expected when state identified students were selected from the top decile and local identified students mainly from the next decile on the achievement and ability-based selection criteria, locally identified students had lower scores on all three achievement measures (correlations ranged from 0.354 to 0.469).

Raven score. The Raven score was related to all three achievement measures, although less so for mathematics (r=0.262) and reading (0.282) than for the 3R test (0.352).

School size. Students in larger schools had higher achievement on all three achievement measures (correlations ranged from 0.145 to 0.194).

AAP percent. AAP students in schools with higher proportions of their students in the AAP, had higher achievement, particularly in reading (r=0.234) and 3R (0.233).

Poverty index. Students in schools in poorer areas, that is schools with higher proportions of their students receiving free or reduced-price lunch, had lower achievement than others, particularly in reading (r=-0.260) and 3R (-0.269).

School ethnic balance. Students in schools with higher proportions of African-American students, had lower achievement, particularly in reading (r=-0.236) and 3R (-0.237).

Retention percent. There was no relationship between the proportion of students retained in grade at a school and achievement of the AAP students at the school. Of course, students in the AAP would normally not be among those not promoted.

Suspension percent. There was no relationship between the proportion of students suspended at the school and the achievement of students in the AAP at the school. Again AAP students would be less likely to be suspended.

It was clearly the case that, with the exception of grade level, retention and suspension, all the student and school variables had lower correlations with mathematics achievement than with either the reading or the 3R tests.

Grade level was particularly unusual in that it was negatively related to mathematics achievement suggesting that students in higher grades had progressively lower achievement, whereas there was virtually no overall correlational relationship between grade and reading or grade and 3R scores. One possible explanation, but one not able to be investigated in this study, is that the MAT7 tests covered skills and knowledge more recently learned by students in lower grade levels, which tended to the negate normal expectation that students in higher grades would have higher achievement.

Explaining student achievement

For the next stage of the analysis a series of multilevel regression equations were developed with students at level 1, grades at level 2 and schools at level 3. The response variables were the MAT7 achievement scores in mathematics, reading and the 3R test. The proportions of residual variance at each of the three levels are shown in Table 5 for the null and full explanatory forms of each model. It will be noted that most of the achievement variance in the null models was at the student level, with totals of only about 10 per cent at higher levels. The even smaller proportions of residual variance at the higher levels when all explanatory variables had been added to the models indicate that the models were more successful at explaining variations between grade levels and schools than between students.

The most important and most consistent variable explaining differences in achievement was the individual student's Raven score. This measure of ability explained 6.6 per cent of the difference in mathematics scores, 7.3 per cent for reading scores and 11.7 per cent for the combined 3R score. Grade level was also consistently and significantly related to achievement across the three tests, but none of the other student or school variables was consistently related to achievement. The variance explained by different sets of explanatory variables for each of the MAT7 scores is shown in Table 6.

Other things being equal, gender was related to both the reading and 3R scores, with girls having higher achievement, but was not related to mathematics scores. Either of the two ethnic variables (individual or school ethnicity) and the school poverty measure provided significant explanation of reading and 3R achievement. However, owing to multicollinearity problems, it was not possible for both ethnic variables to be in the regression equation simultaneously.

Although only the Raven score and grade significantly related to mathematics achievement, the additional effects of the two ethnic variables (one at the individual and one at the school level) and the poverty variable have also been shown in Table 6, for comparison purposes. The maximum variance explained for each of the achievement measures was almost 10 per cent for mathematics, 15 per cent for reading and almost 19 per cent for the 3R test. These are shown in the last row of Table 6.

Comparisons of achievement explanations provided by individual student and school level variables.

It will also be noted in Table 6 that there was little difference between the two ethnicity variables in achievement variance explained, when ability, grade and gender were also included in the regression equation. Perhaps the most interesting result was that inclusion in the regression equation of the school aggregate ethnicity variable, that is the proportion of African-American students at the school, in place of individual student ethnicity, marginally decreased explanation of reading scores (by 0.4%), improved explanation of individual mathematics scores by the same margin, and increased explanation in the 3R test by twice as much. Of course it should be recalled that the two ethnic variables were different in nature. The school ethnicity measure was a proportion, and therefore an interval variable, while the individual ethnicity measure was dichotomous.

It was also of interest that the school-level poverty measure, that is the proportion of students receiving free or reduced-price lunches, explained a little more of the variance in reading and the 3R test than either of the ethnicity variables. And the poverty variable explained the same variance in mathematics as the school ethnicity variable. The combination of the school-level poverty measure and the individual ethnicity variable (together with ability, grade and gender) provided the best explanation for the reading scores (15.0%), although individual ethnicity made no difference for mathematics and virtually no difference for the 3R test when the poverty variable was also included.

SUMMARY AND CONCLUSIONS

When the sources of variance in achievement of students in Years 3 to 5 on the MAT7 tests were considered initially, it was found that almost 90 per cent was due to differences between students, and only approximately four to eight per cent was due to differences between the 30 schools involved. Student-level variables, such as gender, ethnicity and ability, were related to achievement. It was also found, however, that some school-level variables, such as school size, and the proportions of poor students and African-American students at the school, were related to individual student achievement at levels which were comparable with the correlations between achievement and two of the student-level variables, gender and ethnicity. From these results, the potential importance of school-level variables for individual achievement was clear.

When both student and school variables were placed together in three separate regression equations with the MAT7 test scores as successive dependent or response variables, it was found that the maximum amount of variance in individual achievement that could be explained with an optimal set of explanatory variables was between approximately 10 and 19 per cent. The equation with this "best" explanation included the individual student variables of ability, grade, gender and ethnicity, and the school variable labelled "poverty" (the proportion of students at the school receiving free or reduced-price lunches). Two alternative combinations of explanatory variables are perhaps of greater interest given the thrust of this paper. First, other things being equal, the school ethnicity variable provided almost as much explanation of achievement if it was used to replace the individual ethnicity variable. Secondly, once the school variable poverty was in the regression equation, little or nothing was added to the explanation of achievement by the subsequent addition of the individual student ethnicity variable.

These analyses indicate that, at least in this particular situation, the use of aggregated school-level variables for poverty and ethnicity were as efficacious as similar individual student variables in predicting and/or explaining individual student achievement in basic skills areas. As implied above, this finding could have important ramifications for educational researchers interested in investigating home and other background relationships with student achievement but who may have problems in obtaining response rates sufficient to provide accurate and representative individual data. The likelihood is that approval to collect school data will be much more forthcoming and that a set of aggregated social data will thus be more complete than a comparable set including individual student data. In these circumstances, it would seem that nothing need be lost by moving to aggregated data for some socioeconomic and other home background indicators.

The extent to which student and school data and relationships for students in the Advanced Academic Program in one school district in the USA are applicable to the Australian situation requires consideration. For example, ethnicity of students in Australia, both at an individual and school level has different meanings and connotations from those in the USA. The largest ethnic minority groups here are Southern European and, more recently, Asian. The suggestion of at least some of the Australian research, referred to above (p.4), is that ethnicity may not be significantly or consistently linked with achievement in the basic skills.

Perhaps more than ethnicity information, whether individual or aggregated, it would seem that aggregated socioeconomic data has potential as a source of explanatory variables for student achievement. School-based socioeconomic data is not routinely available from Australian schools, however, except for those involved in the disadvantaged schools program or those applying for funding under the program. Consequently it was necessary for Ainley et al (1995) to move to other sources of aggregated information, such as census collectors' districts and postcodes in their exploratory study. The present study adds to the suggestion arising from the work of Ainley et al that, insofar as providing explanation of student achievement, aggregated data are not necessarily a poor substitute for individual data.

Finally attention should be drawn again to the nature of aggregated socioeconomic variables, referred to above (p.3). It was suggested that aggregated variables have a community context. The suggestion is that such a variable, in addition to being the simple aggregate of the corresponding individual student variable, takes on a character of its own which has something to do with the school itself, including the facilites and the views and backgrounds of the staff, and the wider community in which the school is based, and is not simply a function of the characteristics of families of the students who attend. To the extent that the aggregated variable is more than a substitute for the corresponding individual variable, there is the possibility that it will be more closely related to important student outcomes, such as achievement, than it would be otherwise. If our prime concern is providing the maximum possible explanation of variation in student achievement, we may not be too concerned that we do not know the precise nature of the aggregated variable, we may simply be grateful that it does explain as much variance as might be expected from the use of a similar individual student variable.

If, however, the aggregated variable is no more than a coarse measure of the individual student variable, it would be expected that it would contain error based on individual differences that exist between students, and may thus be simply a lesser substitute for the corresponding individual variable. In this case, and in the case where we are concerned with the precise nature of the explanatory variable, we probably need to revert to the equivalent individual student information, and to accept the concurrent loss of sample representativeness resulting from missing data due to a lack of response from parents, either in providing information or in giving permission for their children to provide it.

_________________________________

REFERENCES

Ainley, J., Goldman, J. & Reed, R. (1990). Primary Schooling in Victoria. ACER Research Monograph No. 37. Hawthorn, ACER.

Ainley, J., Graetz, B., Long, M. & Batten, M. (1995). Socioeconomic Status and School Education. Canberra, AGPS.

Bourke, S.F. (1977). Student background and mastery in reading and numeration. In S.F. Bourke & J.P. Keeves (eds), The Mastery of Literacy and Numeracy: Final Report, pp.81-107. Canberra, AGPS.

Clifton, R.A. (1994). Race and ethnicity in education. In T. Husen & T.N. Postlethwaite (eds), The International Encyclopedia of Educational Research (2^nd ed), pp.4891-4896. Oxford, Pergamon Press.

Fuligni, A.J. & Stevenson, H.W. (1994). Home environment and school learning. In

T. Husen & T.N. Postlethwaite (eds), The International Encyclopedia of Educational Research (2^nd ed), pp.2624-2628. Oxford, Pergamon Press.

Go, I.C. (1998). Intra-district mobility patterns among elementary school students in Richland County, School District One. A paper presented at the annual meeting of the South Carolina Educators for the Practical Use of Research, February 23.

Goldstein, H., Rasbash, J., Plewis, I., Draper, D., Browne, W., Yang, Min, Woodhouse, G. & Healy, M. (1998). A user's guide to MLwiN. Multilevel Models Project, Institute of Education, University of London.

Highett, N. (1998). Personal correspondence, 25 November.

Keeves, J.P. (1994). Models and model building. In T. Husen & T.N. Postlethwaite

(eds), The International Encyclopedia of Educational Research (2^nd ed), pp.3865-3873. Oxford, Pergamon Press.

Marjoribanks, K. (1974). The home environment. In K. Marjoribanks, Environments for Learning. Windsor, NFER, pp.99-101.

Williams, T., Batten, M., Girling-Butcher, S. & Clancy, J. (1980). School and Work in Prospect: 14-year-olds in Australia. ACER Research Monograph No.10. Hawthorn, ACER.

TABLE 1. STUDENT PERSONAL INFORMATION BY GRADE

TABLE 2. MEANS (& STANDARD DEVIATIONS) FOR THE INDIVIDUAL STUDENT ACHIEVEMENT AND ABILITY MEASURES BY GRADE

TABLE 3. MEANS (& STANDARD DEVIATIONS) FOR SCHOOL DATA

TABLE 4. ZERO ORDER CORRELATIONS BETWEEN STUDENT AND SCHOOL VARIABLES AND STUDENT ACHIEVEMENT

TABLE 5. UNEXPLAINED VARIANCE AT STUDENT, GRADE AND SCHOOL LEVELS (null model / explanatory model)

TABLE 6. PERCENTAGES OF ACHIEVEMENT EXPLAINED