Abstract:
The construction of a new instrument ought to proceed through a series of sequential stages. Careful documentation of the developing process forms an important and necessary step in formulating a validity argument as evidence to support applications of the new measure. Furthermore, this process has crucial implications for the credibility of any inferences that may be drawn from applying the instrument as an intended measure. Recognising examples of evidence presented the opportunity to appraise the usefulness of the Wolfe and Smith (2007a & 2007b) validity framework which identified seven aspects of validity evidence. The paper presents a summary of the construction of a measure of school need for psychological services and data analyses. Then the activities and findings were assessed as examples of validity evidence with respect to the Wolfe and Smith (2007a & 2007b) framework. In addition, publicly available school-level data were collected such as socio-economic index, suspension and exclusion data, truancy and students with individual behaviour management plans. These were treated as independent variables and the linear measure of school need for psychological services was treated as the dependent variable. Hypothesised associations between the dependent variable and the independent variables were tested by multiple regression analysis of data from 148 school staff members. Each of the school-level indices and the dependent variable were also examined by an estimation of bivariate correlation. The study identified examples of six aspects of validity evidence in the empirical investigation of school need for psychological services. This confirmed the usefulness of the multi-level theory of validity evidence postulated by Wolfe and Smith (2007a & 2007b). In addition, the analyses and graphical displays generated by the RUMM2020 computer program (Andrich, Sheridan, Lyne & Luo, 2003) proved invaluable in illustrating validity evidence. The measure of school need for psychological services was found to be significantly related to student suspensions data. As a result, application of the measure can inform decisions about the level of psychological services that should be provided to schools congruent with the psychological needs of their students.