|
author: Repost
When he was governor of Texas, Bush invited Charles Colson's
Prison Fellowship to start InnerChange Freedom Initiative, a
Bible-centered prison-within-a-prison where inmates undergo vigorous
evangelizing, prayer sessions, and intensive counseling. Now
comes a study from the University of Pennsylvania's Center for
Research on Religion and Urban Civil Society reporting that InnerChange
graduates have been rearrested and reimprisoned at dramatically
lower rates than a matched control group.
How a Bush-promoted Christian prison program fakes success by
massaging data.
By Mark A.R. Kleiman Tuesday, August 5, 2003
The White House, the Wall Street Journal, and Christian conservatives
have been crowing since June over news that President George
W. Bush's favorite faith-based initiative is a smashing success.
When he was governor of Texas, Bush invited Charles Colson's
Prison Fellowship to start InnerChange Freedom Initiative, a
Bible-centered prison-within-a-prison where inmates undergo vigorous
evangelizing, prayer sessions, and intensive counseling*. Now
comes a study from the University of Pennsylvania's Center for
Research on Religion and Urban Civil Society reporting that InnerChange
graduates have been rearrested and reimprisoned at dramatically
lower rates than a matched control group.
For those who know how hard it is to reduce recidivism, the
reported results were impressive. Colson celebrated the report
by visiting the White House for a photo op with the president.
House Majority Leader Tom DeLay issued a triumphal press release.
The Journal smacked critics of faith-based programs for "turning
a blind eye to science" by opposing InnerChange. The report
heartened officials in the four states that have InnerChange
programs and buttressed President Bush's plan to introduce the
Christian program in federal prisons.
You don't have to believe in faith-healing to think that an
intensive 16-month program, with post-release follow-up, run
by deeply caring people might be the occasion for some inmates
to turn their lives around. The report seemed to present liberal
secularists with an unpleasant choice: Would you rather have
people "saved" by Colson, or would you rather have
them commit more crimes and go back to prison?
But when you look carefully at the Penn study, it's clear
that the program didn't work. The InnerChange participants did
somewhat worse than the controls: They were slightly more likely
to be rearrested and noticeably more likely (24 percent versus
20 percent) to be reimprisoned. If faith is, as Paul told the
Hebrews, the evidence of things not seen, then InnerChange is
an opportunity to cultivate faith; we certainly haven't seen
any results.
So, how did the Penn study get perverted into evidence that
InnerChange worked? Through one of the oldest tricks in the book,
one almost guaranteed to make a success of any program: counting
the winners and ignoring the losers. The technical term for this
in statistics is "selection bias"; program managers
know it as "creaming." Harvard public policy professor
Anne Piehl, who reviewed the study before it was published, calls
this instance of it "cooking the books."
Here's how the study got adulterated.
InnerChange started with 177 volunteer prisoners but only
75 of them "graduated." Graduation involved sticking
with the program, not only in prison but after release. No one
counted as a graduate, for example, unless he got a job. Naturally,
the graduates did better than the control group. Anything that
selects out from a group of ex-inmates those who hold jobs is
going to look like a miracle cure, because getting a job is among
the very best predictors of staying out of trouble. And inmates
who stick with a demanding program of self-improvement through
16 months probably have more inner resources, and a stronger
determination to turn their lives around, than the average inmate.
The InnerChange cheerleaders simply ignored the other 102
participants who dropped out, were kicked out, or got early parole
and didn't finish. Naturally, the non-graduates did worse than
the control group. If you select out the winners, you leave mostly
losers.
Overall, the 177 entrants did a little bit worse than the
controls. That result ought to discourage InnerChange's advocates,
but it doesn't because they have just ignored the failure of
the failures and focused on the success of the successes.
The Penn study doesn't conceal the actual poor outcome: All
the facts reported above come straight from that report. But
the study goes out of its way to put a happy face on the sad
results, leading with the graduates-only figures before getting
to the grim facts. Apparently, the Prison Fellowship press office
simply wrote a press release off the spin, and the White House
worked off the press release. Probably no one was actually lying;
they were just believing, and repeating as fact, what they wanted
to believe. It's hard to know for sure what those involved were
thinking: Study author Byron Johnson canceled a scheduled interview
at the last moment. The White House didn't respond to requests
for comment.
InnerChange program manager Jerry Wilger says he doesn't know
much about research, but he doesn't think it's fair to count
the performance of the people who dropped out of his program
against him, a fair-sounding objection that misses the point
entirely. If InnerChange's 177 entrants were truly matched to
the control group but ended up having more recidivism, then either
the apparent success with the graduates was due to "creaming"
or the program somehow managed to make its dropouts worse than
they were to start with. If the program genuinely helped its
graduates and didn't harm its dropouts, and if the whole group
of entrants was truly matched to the controls, then the group
of 177 should have done better than the controls. And they didn't.
So, the feel-good winners-only analysis simply isn't worth
the paper it's printed on. Only the full-group analysis (known
technically as "intent-to-treat," a holdover term from
its origins in medical research) has any real value. And on that
analysis, the program has a net effect of zero or a little worse
than zero. That makes it a loser.
John DiIulio, an intellectually serious advocate of faith-based
programs who was the first director of the Bush administration's
faith-based initiatives and the founder of the Penn research
center, acknowledges frankly the results weren't what a supporter
of such programs would have hoped for. But he points out that
a single study almost never provides a convincing yes or no answer
on a program concept. "The orthodox believers point to a
single positive result and say it proves faith-based programs
always work. The orthodox secularists point to a single negative
result and say it proves faith-based programs never work. They're
both wrong."
The poor result of InnerChange doesn't mean that no faith-based
prison program could work, but it does mean that this one hasn't,
at least not yet. It joins a long line of what seemed like good
ideas for reducing recidivism that didn't pan out when subjected
to a rigorous evaluation. Maybe my own pet, literacy training,
wouldn't do any better in a real random-assignment trial. But
that's why you do evaluations; they tell you things you didn't
want to hear. If you're honest, you listen to them.
And if you're smart, you don't listen to the political advocates
of "faith-based" this and that when they say they're
only asking us to support programs that have been "proven"
to work.
-Correction, Aug. 6, 2003: The article originally and incorrectly
described InnerChange Freedom Initiative as "a fundamentalist
prison-within-a-prison." The Prison Fellowship regards itself
as being part of the evangelical tradition rather than the fundamentalist
tradition. (Return to the corrected sentence.)
Source: http://portland.indymedia.org/en/2003/08/269386.shtml |