Information and communication technologies (ICT) have changed, and continue to change, the education, work and social lives of people. Many education authorities have come to see competence in ICT as a key preparation for young people's futures and several have established assessment programs in this field. It is expected that students will become more proficient in their use of digital technologies as those technologies become more frequently used and as they learn to use those technologies. However, the measurement of those changes is complicated by concomitant changes in the technologies used. This paper is based on data from a computer-based assessment of ICT Literacy in Australia conducted with large nationally representative samples of approximately 10,000 students Grade 6 and Grade 10 students in 2005, 2008 and 2011. It describes methods used for equating those assessments and measuring change over the period. ICT Literacy is seen as a set of generalisable and transferable knowledge, skills and understandings concerned with the use of computer technology to investigate, create and communicate information in a variety of contexts. The assessments combined the performance of specific software functions with the creation of digital products. Each assessment involved tasks embedded in seven thematic modules. In 2011 six of the modules were built around a narrative and required the production of a digital product and one focussed on the performance of discrete computing tasks. The six thematic modules involved using blog web-sites, search engines, graphics software, mapping software, photo management software, producing a video, editing formatting and updating a wiki, and using collaboration software to work on a project with others. IRT methods were used to analyse the student responses and generate a scale for locating items from each module and reporting student achievement. The scale was highly reliable (0.93), and was characterised by descriptions of proficiency levels based on item difficulties. Common tasks were used to compare the relative performance of the Grade 6 and Grade 10 students and other common tasks were used to link the 2011 results to those from 2008 and 2005 (there were three modules from previous cycles and four new modules). The results show an improvement in the ICT literacy among Grade 6 students but no change for Grade 10 students. In addition the results indicate that ICT literacy is related to student perceptions of ICT (interest and ICT self-efficacy) and their socioeconomic background.