Jan. 22 - Inside Higher Ed
Bias Seen in Bias Studies
Professors are all Democrats, except those who are communists.
Professors all hate Bush. Professors favor like-minded students and love
converting those who love God, country and the president. You've read
all the claims and more, in right-leaning blogs and columns. Frequently,
these claims are based on studies - many have been released in the last
two years - of professors. Party registration is documented, or
professors respond to surveys, or syllabus content is rated.
A new study being released today aims to debunk all of those studies.
"The 'Faculty Bias' Studies: Science or Propaganda," takes eight of the
recent studies on faculty politics and judges them by five general tests
of social science research. Today's study finds that the eight all come
up short in adhering to research standards. The new study was sponsored
by the American Federation of Teachers and the work was conducted by
John B. Lee, an education researcher and consultant who said that once
the AFT commissioned the work, it did not restrict his approach or
findings in any way.
The various studies analyzed are by no means identical, but they tend to
have two major themes (although some stress just one of the themes):
that faculty members are liberal and that their liberal inclinations are
significant in considering their performance.
Lee's analysis finds some support for the first theme. "Taken together,
these studies at best suggest that college faculty members are more
likely to be Democrats than Republicans," he writes. However, even on
this theme, he notes that the studies tend to exclude community college
faculty members and to focus on faculty at elite institutions - probably
skewing the results.
The second theme takes a more thorough beating in the study. "Among the
most serious claims the authors make is that this liberal dominance
results in systematic exclusion of conservative ideas, limited promotion
opportunities for conservative faculty, and expression in the classroom
of liberal perspectives that damage student leaning," Lee writes. "These
claims, however, are not supported by the research. Basic methodological
flaws keep a critical reader from accepting the conclusions suggested by
The flaw Lee identifies most frequently with this theme is one in which
researchers note a correlation and - in Lee's opinion - then see a
causal relationship without sufficient evidence that one exists.
AFT officials said that they commissioned their study out of concern
that the drumbeat of reports on political bias were suggesting to the
public and politicians that faculty members are unprofessionally
injecting politics into the curriculum, hiring and grading. Some of
those whose work is criticized in the AFT's report, however, said that
it was the faculty group's report that was guilty of bias, and they
questioned the legitimacy of the new study, which they termed
The New Research
The new AFT study looks at eight studies, including some that have
attracted substantial attention (both praise and criticism), such as
work published in 2005 in The Forum that analyzed faculty attitudes at
four-year institutions and concluded that conservatives, practicing
Christians and women are less likely than others to get faculty jobs at
top colleges. That study was based on a survey of 1,643 faculty members.
Other studies looked at faculty attitudes in certain disciplines or at
Some of the studies were prompted by specific events, such as the
American Council of Trustees and Alumni's "How Many Ward Churchills?,"
which analyzed class materials online at top institutions and found that
the controversial Colorado professor's ideas - which have been in the
news while his university has considered whether to fire him - are
shared by many professors. Some of the reports are by social scientists,
published in peer-reviewed journals. Others were issued by associations
that are players in the culture wars of academe.
Lee said that to test the validity of the studies, he wanted standards
that could not be considered partisan, so he used a 2006 statement by
the White House Office of Management and Budget about objectivity in
research. Based on that statement, he asked five questions about each of
the faculty bias studies:
* Can another researcher with a different perspective replicate the
results using the information provided by the author?
* Are the definitions used in the studies clear enough?
* Does the research eliminate alternative explanations for the
* Do the conclusions follow logically from the evidence?
* Has the author guarded against assumptions that could introduce
systematic bias into the study?
Using this framework, Lee gives the studies failing grades. Four studies
had data that could be replicated, and he gave three studies acceptable
reviews on clarity of terms, but it was downhill from there, and he
argues that none of the reports can truly back up their contentions.
Besides offering that general rubric, Lee goes through each study,
summarizing problems he found with it. For example, "How Many Ward
Churchills?" was based on a review of online materials at various
colleges. Lee notes that the researchers for the study appeared to focus
on syllabuses or courses that had certain key words: activism,
discrimination, gay issues, Marxism, oppression, pornography, radical,
women's studies, among others. Lee writes that selecting 65 courses at
48 colleges "does not allow for the sweeping generalizations the authors
Even for those courses, he notes, the authors of "How Many Ward
Churchills?" didn't actually observe the courses, so while they may know
that certain topics or perspectives are covered, they have no way of
judging the intellectual character of a classroom. While that report's
authors wrote that Americans should be "outraged by the one-sided
doctrinaire perspective" of their courses, Lee writes that they had no
evidence to assert much of anything about the courses.
In several of the studies, Lee notes that relatively small subgroups of
college faculty were surveyed, generally professors at elite and/or
four-year institutions. Because community college professors, on
average, are more centrist and more religious than colleagues at
four-year institution, Lee questions whether their exclusion limits the
ability of the studies' authors to make statements about academe as a
Another theme he returns to over and over again is one of demonstrating
(or not) causal relationships. He notes that there are many explanations
for political trends and demographics among the professoriate, so it is
unfair to assume that a liberal tilt (assuming one exists) reflects
bias. He notes, for example, that the studies do not explore whether
there could be non-political explanations.
Many have questioned, for example, the lack of data on applicant pools
for faculty positions, and compared the disparity in political
inclinations to that of Wall Street, where there are not suggestions
that any Republican tilt is the result of bias or results in any
discrimination against Democratic investors. Lee also compares the
military, where recent polls have found a Republican tilt in opinions,
but no evidence that soldiers service to their country is affected by
whether they are seeking to protect members of one party or another.
While Lee finds flaws in all of the studies, he says that they have had
influence, and notes that the studies have been widely cited by
conservative pundits. Looking at the studies together, he says that it
is clear that the authors "have a clear agenda" of charging professors
with unprofessional conduct, and yet lack the evidence to make their
case. Not a single study, he says, shows political bias in the classroom
or hiring decisions.
"Until credible studies are conducted to provide a more grounded and
systematic approach to understanding the subtle relationship between
political beliefs and professional responsibilities, it is irresponsible
to suggest that the conclusions reached in these reports represent a
scientifically derived set of facts. They do not," Lee writes. "Passing
off personal opinions as facts is not science; it is the antithesis of
what serious researchers try to do, regardless of whether they are
conservative or liberal."
Two of the authors whose work is criticized by the AFT took issue with
the conclusions, and questioned whether the organization could fairly
look at these issues.
"Critical commentary is always socially useful, and this new report is
no exception. Even just a cursory reading will teach us much about the
moral and intellectual character of its sponsors - the AFT, the AFL-CIO,
and Free Exchange on Campus," said Daniel Klein, a professor of
economics at George Mason University and the co-author of two of the
studies reviewed in the report.
Anne Neal, president of the American Council of Trustees and Alumni
(which issued two of the reports reviewed), criticized the AFT for
commissioning the study. Via e-mail, she said: "Faced with mountains of
evidence from ACTA and others documenting a troubling lack of
professionalism in the academy, AFT chooses, instead, to shoot the
messenger. In doing so, far from undermining ACTA, it discredits itself.
AFT's study is severely flawed. It is filled with inaccurate and
tendentious interpretations - for instance, framing the debate in terms
of politics rather than professional standards outlined by ACTA;
applying irrelevant 'scientific' standards to textual analysis; and
offering such shoddy research that the sections on ACTA totally confuse
and conflate two different reports, rendering the critique invalid, even
She added: "In the face of troubling evidence of a politicized
classroom, has AFT conducted any studies of its own to see if there is
problem? Taken concrete steps to explore the atmosphere in the
classroom? The answer, of course, is no. AFT's report is not science -
With more studies of the sort the AFT criticized in the works now, stay
tuned for more debate.
- Scott Jaschik