James J. Heckman is the Henry Schultz Distinguished Service Professor of Economics at the University of Chicago. He shared the 2000 Nobel for his work on correcting for selection biases when doing econometric studies, developing techniques which he applied to measuring everything from the economic effects of civil rights laws on African-Americans to the economic benefits (or lack thereof) of GEDs. Recently, he has done considerable work on early childhood education, including detailed studies of the Perry preschool experiment; Wonkblog interviewed him in 2010 on that subject.
We spoke on the phone Wednesday night. A lightly edited transcript follows.
Dylan Matthews: The Abecedarian and Perry experiments provide pretty definitive evidence that individual preschool programs have strong effects, but Obama’s been touting certain statewide programs like Oklahoma’s. What’s the evidence for those like?
James Heckman: I would be cautious. I’m instinctively cautious because I’m an academic. The Perry and ABC (Abecedarian) and some others, the nurse-family partnership, have not only had randomized trials, but have also followed people up for decades. The Perry people are now 50 years old. The ABC people, now they’re close to 40. We actually can follow them in a way that the other programs don’t follow their participants. The state programs have relatively short-term evaluation plans.
And I think, you’re right, they’re not randomized controlled trials, so I’m a little cautious. I don’t find them as convincing. As far as I know they’re not of the same quality. I have not personally relied on them. That’s not to say they’re bad programs, they just haven’t been evaluated as thoroughly.
Look, the program that has been promoted strongly in Oklahoma, I do know and it’s Educare, and it was primarily put forward by George Kaiser. And they’re just now conducting evaluations of Educare, and I know because it’s being conducted here in Chicago, by people I know.
I didn’t think the evaluation I saw was particularly good. There was an evaluation that was highly questionable, and if I told you what it was, you wouldn’t be too impressed. They took kids who were taken into school one year earlier than others, and used a technique called regression discontinuities. You look at birthdays. So they compared people who were just on one side of the boundary between grades to the other side. You can use that as a pseudo-experiment. He followed these kids out one year, and the ones who didn’t get in hadn’t been taught reading, so the real question is what they would have been like in five years. It kind of left me with a cloud of suspicion.
Dylan Matthews: High-quality preschools, like those in **Perry and Abecedarian, tend to require big upfront investments. Abecedarian was about $19,000 a year per student. Less effective programs, like Head Start, cost a lot less upfront. Do the results worsen the lower that number goes?
James Heckman: The Perry study would be lower, about what you spend on high school students in an average quality high school. ABC was more expensive. ABC also was lasting for about eight years. It took kids from birth, more or less, through third grade, and it was considerably more expensive. So [money] is quite relevant.
Head Start, I think, actually comes out better, will come out better. There probably is a real dimension here. We had a governor here in Illinois you might have heard of, Blagojevich, and I met with him and he was very inspired by the work we were doing, and put forward a program here in Illinois, but it had the feature of being very diluted, and I think that’s the danger.
Quality really matters. That’s been pretty well documented. I would argue Perry, which has been most thoroughly evaluated, is extensive. In terms of the return on investment, per dollar return, the annual return for what you’d get on a bond or some kind of fixed income, you would have a rate that was 6-10 percent per year, which is extremely high. So even though it costs something, it’s about the return is to society and to the individuals.
They are very good investments. They’re very comparable with stocks at the end of the second World War. Return was about 6.9 percent. Pretty comparable. It’s a range, because there are certain subjective elements. But that’s a very high rate of return and it’s far superior to a range of activities, compared to, say, Job Corps, where the return is negative. I’m an economist. I would talk about both the benefits and the costs. And if the benefits really outweigh the costs, I think that’s something very rare. So it’s a good investment.
Dylan Matthews: One thing a skeptic could say is, “Sure, small sample pilot programs show these great results, but randomized evaluations of whole programs haven’t happened.” What do you make of that critique?
James Heckman: Charles Murray has made that claim most recently, and others make it too. Two things. A small sample would actually work toward not finding anything. You have a limited number of observations. You would argue that the statistical observations would not be very great, and there would not be much of them. There are methods that account for the small sample size. Size doesn’t matter. It holds up. There’s a lot of robustness here, in the sense that you can find the same patterns in other areas, like in programs in Jamaica.
There are examples of government programs that seem to have some promising results. The most prominent is the Chicago parent-child program. It wasn’t evaluated by a randomized trial, but the evidence there is quite promising, and showed effects that last until adulthood. That’s been done in a very disadvantaged areas, and there have been a lot of tries, to kick the tires and really test it, and it seems to hold up.
But there’s always an issue with these pilot programs. Maybe these are just highly motivated people who would do a pilot program. I think you were going to say something about Head Start having a mixed effect, as the most recent evaluation found. That shouldn’t be taken too seriously for two reasons. There’s a real problem with the way the evaluation study was conducted, and it’s a very low quality program. If the quality of the program goes down, then you’d really have to be careful. I couldn’t give you a monetary threshold, but they can do harm. Parents can put them in mediocre child care environments, away from the stimulation they might get at home.
Dylan Matthews: We touched on Charles Murray, so here’s another critique from the symposium you did at Boston Review. The Harvard Graduate School of Education’s David Deming has argued, and has a study on this, that Head Start gets 80 percent of the gains of Perry. And it’s much less expensive. I’m guessing you’re skeptical of that finding.
James Heckman: Well, the reason why I’m skeptical is that the most salient work on Head Start is this new evaluation which came out last October. It actually came out later than I responded to Deming. I am skeptical for the following reason. It’s really heterogeneous, and I’m sure there are some very high quality programs and some very weak ones. The latest study showed very weak effects. That was a short-term followup. Head Start has never had a long-term followup.
I know Deming’s work and other work, even from former students of mine. Jens Ludwig had something — there are several papers that are showing very favorable Head Start effects. It’s very mixed, it’s not like it’s universal across groups. And all of these methodologies, the question is how robust the results really are. I’m skeptical that you can get by on the cheap, really cheap, with Head Start. Head Start doesn’t have one curriculum. Some Head Start programs are really using a version of Perry, maybe a diluted version. But others are using other curricula. So you wouldn’t expect to see the same effects across these programs. The quality of inputs is lower. Maybe Deming’s right, I think it’s worth studying more. But I am skeptical. It’d be wonderful if he were right.
Dylan Matthews: Really, there are Perry replicas in Head Start? I’m a bit embarrassed I didn’t know that.
James Heckman: It’s not a complete replica. But it’s patterned after that program, I wouldn’t say closely patterned, but patterned. About 30 percent use that. Not a replica in terms of spending, but that use teachers in the Perry way. The people in Perry don’t think it’s valid to use it as a replication of what they do but if you look at it they really do use it.
Dylan Matthews: Do we know if those programs are better than regular Head Start programs?
James Heckman: I do not know that, to be precise. It sure costs a lot less. That’s why the Perry people are probably skeptical that it’s a true implementation, but it follows the guidelines.
The recent Head Start evaluation, I think it was a little broadly drawn. I would say it would be nice to me if we had an explanation of why the initial effects are so discouraging. I think I know why, that it’s lower quality, but it’s a question.
Dylan Matthews: It does seem that if Deming’s right, that would reduce the harm that state budget cuts could do to program quality.
James Heckman: I would turn it around. If you look at the reduced costs, that’s just the cost upfront. Reduced crime, reduced health care costs, lesser burden on educational systems, even in elementary school — you see payoffs coming all along the way, and those are all outcomes the government is financing, so it’s going to save money, and not in the too distant future. Especially if elementary schools start having special education.
I just kind of am asking for people to think about this more comprehensively, to actually look at the structure of the whole plan. Another study that’s really interesting is what if you compare reduced classroom sizes to something like Perry. You get a substantial return to Perry but small class sizes, it’s essentially nothing. There’s a political economy explanation that all areas of government are dirty, but I think the benefits are real and we won’t have to wait twenty years to see them. But it is an investment, like an airport or dam.
Dylan Matthews: To circle back, one criticism of Perry’s applicability in other contexts is that it had an IQ ceiling. Students had to have IQs of 80 or under. So you could argue that it shows that preschool can help those with lower IQs, but not students generally. Abecedarian didn’t have that problem, but what do you make of that critique?
James Heckman: First of all, we actually did a study which actually looks at how representative it is. Don’t forget, the Perry targeted population is the larger African-American population, so that’s an issue. At the time that Perry was done, the average IQ of an African-American was 85. That’s since improved. So there was a gap, no question. Part of the funding requirement was that people in the program have an IQ below 80, thought that wasn’t explicitly enforced.
Where they’re more atypical is in the level of disadvantage. You’re talking about people whose income is low, so you’re getting the 16 percent of the African-American population that’s most disadvantaged. The IQs were about a third of a deviation below the mean, if anything it’s not even a third. Don’t forget, we’re talking about IQ at enrollment, and it fluctuates. It’s at age 10 that it stabilizes. So I don’t think it’s that atypical of the African-American population at the time.
ABC is actually above normal. The kids were basically infants, and they couldn’t measure an IQ at six weeks. What they did is have measures of parental disadvantage. And the kids are measured at the same age as Perry, and you’re getting IQs in the high 80s and the early 90s. I completely agree that ABC has an advantage on that.
A lot of the evidence, almost all the targeting is for disadvantaged people. The strongest evidence for strong returns is among the disadvantaged. So anybody criticizing it is criticizing the idea that you might want to make it universal. Middle class families might not have same benefit. So that’s why you’d bring this up.
What’s interesting is that measures of IQ are showing convergence. It used to be that black IQs were one standard deviation below white IQs. It’s no longer one standard deviation. The average is 100. The black population was one standard deviation below and now it’s about 2/3. I don’t monitor these things daily, but I see relatively recent progress. That goes to show that these gaps can be fixed.
Source: Washington Post – http://goo.gl/yQFPr
