Today’s email brought a content alert from Economic Inquiry on a newly accepted paper titled “New Evidence on National Board Certification as a Signal of Teacher Quality”. The abstract of the paper reads:
“Using longitudinal data from North Carolina that contains detailed identifiers, we estimate the effect of having a National Board for Professional Teaching Standards (NBPTS) teacher on academic achievement. We identify the effects of an NBPTS teacher exploiting multiple sources of variation including traditional-lagged achievement models, twin- and sibling-fixed effects, and aggregate grade-level variation. Our preferred estimates show that students taught by National Board certified teachers have higher math and reading scores by 0.04 and 0.01 of a standard deviation. We find that an NBPTS math teacher increases the present value of students’ lifetime income by $48,000.” (emphasis added)
Based on the abstract, one might infer that having NBPTS certification makes for a better teacher and that having NBPTS certification allows math teachers to have a meaningful lifetime income effect on students. If you read just a bit further, you might feel comfortable that you made the right inference when you read:
With aggregation and school-by-year fixed effects only variation between cohorts is used to identify the effect of NBPTS on test scores.
Unfortunately, if you concluded that being NBPTS certified has a meaningful relevance to student performance, you’d be completely wrong–as the paper itself explains.
Reading the abstract, the first question one should ask is “How does one become NBPTS certified, and what does that have to do with teacher quality?” In statistical terms, there’s a significant question of endogeneity and causation. Namely, does getting certified make one a better teacher, or do only better teachers get certified? If the latter, then whether or not one is certified has nothing to do with academic outcomes. It might provide a signal that the teacher is already a good teacher, but having the certification would have no meaningful effect on academic outcomes or lifetime earnings.
And indeed, that is exactly the case, as the authors themselves explain if you read just a little further than the statement about their method “to identify the effect of NBPTS on test scores.” What matters is the teacher and the teacher’s skills and practices. The certification itself is superfluous to the academic achievement result.
“Comparisons of teacher performance before and after certification suggest that greater average effectiveness of certified teachers reflects fixed quality differences identified by the certification as opposed to human capital effects. Implementing policies with a primary goal to modify the effectiveness of teachers should place little weight on the NBPTS certification as a potential facilitator. Rather the certification can be used to reward more effective teachers where use of direct evidence on performance in the districts is not feasible.” (emphasis added)
In other words, it’s the teacher-effect, not the NBPTS effect, that matters–to the point that the authors specifically say that little weight should be place on NBPTS certification as a potential facilitator (i.e., a policy tool) for improving student outcomes.
What the authors really purport to show, as the paper title alludes, is that being NBPTS certified is a pretty good indicator that a teacher is a good teacher. The NBPTS standards appear to be well-aligned with effective teaching practices. But if that’s the actual research objective, then the authors should also have looked at the causation from the other direction and tried to sort out the selection bias in who wants to get certified and why.
It’s unfortunate that the abstract of the paper is so misleading, because many people economize on their time by only reading the abstracts of articles to get a sense of the paper’s results. After all, that’s the purpose of an abstract. In this case, however, the abstract is written so poorly that it buries the actual results beneath a misleading presentation and might be perceived as a serious case of ‘bait and switch’. While the title of the paper is still correct–having national board certification is a signal of teacher quality–the abstract’s wording risks painting a false picture of the relevance of certification for student achievement. And that false picture is perpetuated by their description of their methods.
The editors of Economic Inquiry should be a bit ashamed for allowing such a bait and switch. The authors should as well. At best, it’s carelessly poor writing. At worst, it’s the academic equivalent of click-bait. Unfortunately, some people may look no further than the abstract as the basis for what would ultimately be a misguided potential education policy.