Formative Ed Research, Part 2. The Abecedarian Project
Part 1: The Coleman Rorschach
Previously we reviewed the Coleman report, the first large-scale data-driven study of inputs versus outputs. Coleman showed that varying outcomes seemed to have no connection with school quality.
Mostly, Coleman’s lessons were ignored. The primary question — how does one impact outcomes? — begs another question.
I’ve spent the better part of my career in assessments (fun, I know) — designing, developing, building the delivery platforms, etc. Most people, particularly not knee-deep in the assessment world, focus on what’s on the test? and what does my child need to learn? and what’s a passing score? Obviously there are questions of how are the questions constructed and what are the scoring parameters?
But consider: if I teach you something Monday morning and test you Monday afternoon, you might do fairly well. But what if I test you on Tuesday? Or the following week? Or the following month or year? If I’m assessing you a year later, it almost doesn’t matter how complex or basic the lesson was, you’ll probably fail.
This leads to the question of what’s the point of the desired outcome? Are we trying to build life skills or improve cognitive function or are we just attempting to teach students about how a bill becomes a law? If we’re actually trying to teach larger skills, then it makes sense to assess them later — how much later is entirely another question.[1]
So let’s look at two projects that had longitudinal assessment models.
The Abecedarian Project was a project started in 1972 that took 57 infants from severely disadvantaged backgrounds and provided intensive (e.g. 6+ hrs/day) cognitive enrichment for about five years. Most of the enrichment was language/game-based. The idea was that with such cognitive stimulation and so early (the typical enrollee was only a few months old at start) and for such a long period of time (again, 5 years), the outcomes would be dramatic and at a cognitive level (you know, like Baby Einstein promised). There was a control group of 54 infants selected from the same population. Both groups were provided nutritional support to negate any effects nutrition may have on cognitive development.
Initially — generally within six months — the Abecedarian Project seemed like it was producing amazing results. The participants were tracked and assessed repeatedly (most recent assessment was at age 30). The amazing cognitive results that the program seemed to be producing in its first few years had largely disappeared by age 21. While participants did score higher on IQ tests, they still averaged in the 25th percentile (the control group was in the 16th percentile). Any honest psychometrician would confess that, on an individual level, such a difference is meaningless and would have no impact on practical outcomes.[2]
The Abecedarian Project demonstrates an obvious issue. When tested in the middle of the program, the students seemingly performed far above average. But by adulthood, the substantial additional efforts in early childhood seemed to produce negligible long-term cognitive differences. We all know that most students forget facts quickly. But if we’re teaching them to improve general cognitive skills, then, presumably, those skills produce positive results in adult life. Is there evidence this is possible?
The other interesting factor with the Abecedarian Project is that the participants do seem to have improved non-cognitive outcomes: more attended college, held down jobs, stayed off public assistance, etc. So the positive impact seems to be behavioral, perhaps even cultural. This would somewhat align with Coleman, which identified positive outcomes with parents, not schools.
By the mid-1980s, another program called The Infant Health and Development Program (IHDP) was launched to test the Abecedarian model on a larger scale. IHDP focused on low-birthweight babies and had an experimental group of 377. Like Abecedarian, IHDP’s initial results for children at ages 2 and 3 were impressive; their cognitive development was demonstrably superior to that of the control group. But by age 5, the differences disappeared. More assessments at age 18 showed that the experimental group did not demonstrate cognitive improvements.
One of the lessons of these (and similar) studies if that if one wants to discredit a ‘cognitive improvement’ study, just wait a few years. Whatever improvement that was detectable (per an assessment during or recently after the program) will be largely gone.
Back to the question: how does one impact outcomes?
If the desired outcomes are cognitive or content-based, the research isn’t very encouraging for longitudinal improvements that have a practical impact on individuals. However, if the outcomes are behavioral, then perhaps there’s some hope.
Part 2/x. Next: Kansas City.
[1] A related question not addressed here is whether the test is attempting to purely assess or also teach. Tests are actually good teaching tools (because of the emotive state of the test-taker), so using tests as teaching tools can be quite effective (cf. Asian education models).
[2] I should point out that the Abecedarian Project has had much controversy over the way its findings have been produced, from basic issues such as the mysterious differences in the number of participants to the methods for producing results. Some suggest that the results have been rigged and may not be as positive as reported.
About Nathan Allen
Formerly of Xio Research, an A.I. company. Previously a strategy and development leader at IBM. His views do not necessarily reflect anyone’s, including his own. (What.) Nathan’s academic training is in intellectual history; his next book, Weapon of Choice, examines the creation of American identity and modern Western power. Don’t get too excited, Weapon of Choice isn’t about wars but rather more about the seeming ex nihilo development of individual agency … which doesn’t really seem sexy until you consider that individual agency covers everything from voting rights to the cash in your wallet to the reason mass communication even makes sense….
