On the AMCHA "Bringing BDS into the Classroom" Study

The AMCHA Initiative has a new self-published study, which is gaining some attention in the Jewish Press, purporting to show that when teaching courses related to Israel, Palestine, or Zionism, (academic) BDS-backing faculty are more likely load up their syllabi with fellow BDS-supporters, as opposed to giving their students a balanced perspective. Their big takeaway findings are:

• For academic BDS supporters, a median 78% of syllabi readings are by "BDS supporters", compared to a 17% for non-BDS supporters.
• All academic BDS supporters have  at least a majority of readings from "BDS supporters", whereas only 6% of non-BDS supporters have a majority of readings from "BDS supporters".

I'd been hearing about this study for awhile, and I was finally prompted to look it over after reading this laudatory account from Daniel Gordon. Gordon argues that the AMCHA study proves that BDS supporters are sabotaging the values of balance and even-handedness in favor of a one-sided propagandistic approach in their classrooms -- so much so that he suggests that either academic administrators or "the public" at large might be justified in interfering with their content.

That's an aggressive claim, and one that requires equally strong proof. Yet the AMCHA study provides, at best, mixed evidence regarding who is actually being one-sided. Let me present AMCHA's data in a different way that might generate a different intuition (this is derived from "Figure 1" of the study):

• 63% of "no BDS" faculty have at least 80% of their syllabus readings come from fellow "no BDS" authors. For 31%, that number rises to 90%.
• By contrast, just 33% of academic BDS supporters have at least 80% of their syllabus readings come from fellow BDS supporters. And only 7% (aka, one syllabus) has over 90% of readings come from other BDS supporters.

Framed that way, one could argue that it's the BDS supporters who do a better job avoiding overwhelmingly one-sided syllabi. Two-thirds of them devote at least a non-trivial chunk of their syllabi to their ideological opponents. By contrast, BDS opponents are far more likely to completely or almost completely load their syllabi up exclusively with fellow BDS opponents. If the idea is, in Gordon's words, to present "competing narratives", it's far from clear that BDS opponents are doing their diligence in meaningfully presenting the "opposing side".

Now to be clear, I think one can very easily overinterpret this framing as well. Most notably, it doesn't take into account the base rate -- what percentage of authors active in Israel Studies qualify as "BDS supporters"? If it's only a marginal few, then the "no BDS" crowd might be giving their views proportionate weight even if they only occupy 10 - 20 % of the syllabus (and this would, similarly, suggest that BDS backers are significantly oversampling their side relative to its support levels). But base rate levels are very difficult to tease out, and almost certainly vary depending on the specific topic of the course. It's quite plausible that many more authors working on "Palestinian Literature of Resistance" support BDS compared to those working on "Israeli Constitutional Law", and so the fact that a class on the former contains many BDS-backers on the syllabus (or one on the latter contains few) does not necessarily reveal any especial bias on the part of the professor. And that doesn't even go into the over-simplified notion that "BDS support/opposition" necessarily mark out the relevant ideological "sides" for every class.  The difficulties in parsing these out in practice, when they're almost always going to be matters of interpretation, are just some of the many reasons why outside observers should be very leery about substituting their own judgments regarding "balance" and "even-handedness" for that of the professor.

The AMCHA study also raises another pertinent question: Who counts as a "BDS supporter"? Here, the study authors do something rather slick -- they change their answer for the independent versus the dependent variable. The result is to significantly inflate the level of same-side bias that exists on the side of the "BDS supporters".

Here's how it works: For the independent variable -- that is, the professors authoring the syllabi -- the study only includes supporters of academic BDS, specifically, as "BDS supporters". On the other side, "no BDS" professors are those who oppose BDS in all its forms (not just academic BDS but, say, settlement boycotts as well). Those who fit into neither group (people who support some forms of BDS, but not academic boycotts) are excluded.

But for the dependent variable -- that is, which syllabus readings are identified as being authored by "BDS supporters" -- that entire middle-ground category is grouped back into the pro-BDS crowd. Along this axis, "BDS supporter" includes not just academic boycotters, but also

• Anyone who has supported, or signed a petition in favor of, any form of BDS (including settlement boycotts);
• Anyone who has supported a boycott of "specific Israeli leaders"; 
• For Israeli scholars, anyone who either has "challeng[ed] Israel’s right to exist as a sovereign Jewish state" or "call[ed] on Israelis to resist obligatory military service", regardless of whether they've endorsed any form of BDS or not.

And any reading that is even co-authored by anyone in this expansive category is coded as being by a "BDS supporter".

Putting these two different renditions of "BDS supporter" together, the study is measuring the homophily of what we think of as the hardest core of BDS supporters (those favoring academic boycotts) by asking how many of their syllabi readings are at least co-authored by anyone at least as left-wing as Peter Beinart. This is not, to say the least, an especially illuminating metric.

In short, the AMCHA study fails to demonstrate that pro-BDS faculty produce syllabi that are more one-sided than their anti-BDS peers. Indeed, it may instead offer some (albeit weak) evidence in the opposite direction. Not only does it almost certainly exaggerate the same-side bias of BDS supporters by contracting and then expanding the definition of "BDS supporters", but even on face the study findings seem to show that the most ideologically-lopsided syllabi -- in terms of giving significant attention to authors on the opposite side of the BDS ideological divide -- tend to come from BDS opponents.