The Ever-Changing NAEP Sample

Matt DiCarlo:

Next, by grade: In 1978, 28 percent of NAEP-LTT test takers were in 7th grade or lower, compared with 39 percent in 2012. Although standards and curriculum are different today, it's worth noting that the 13-year old sample has changed as far as where they are in the K-12 system.

Third, there is the difference in parental education. The proportion of the 2012 sample with parents who completed college is over twice as high (54 percent) as in 1978 (26 percent). Conversely, the percentage of 13-year olds with parents who have a high school diploma or less is half its 1978 level. Again, some of this change is recent - for example, the proportion with a high school diploma or less was 27 percent in 1999, compared with 20 percent in 2012.

In short, the student population, and thus the NAEP samples, are changing, over the short- and longer terms. Any concurrent changes in testing performance may just as easily be due to these and many other shifts in the characteristics of the test takers- including unobservable factors that cannot be gleaned from breakdowns by subgroup - as to any change in school performance. This most certainly does not mean that schooling quality is unimportant, only that raw NAEP scores by themselves do not measure it very well, and they're not supposed to.

Posted by Jim Zellmer at July 4, 2013 12:38 AM
Subscribe to this site via RSS/Atom: Newsletter signup | Send us your ideas