Jeffrey Toobin notes that taking "diversity" into account when making Supreme Court nominations has a deep pedigree in American history. First it was regional diversity. Then it was religious (Catholic and Jewish) diversity. Now, racial and sexual diversity are paramount. And these roughly track the evolution of what differences are politically salient in American society (or more accurately, at what point in our nation's history "different" marginalized groups were able to effectively challenge their political exclusion).
There's nothing strangely judicial about this either. Matt Yglesias notes that cabinet appointments, too, have historically been highly keyed to diversity considerations. Remember the fit southerners pitched when they thought that Obama wasn't going to have any (White) dixie residents in his cabinet? At the founding, the Federalists were quite keen on arguing that diverse representation was both key to the success of the American project, and provided (among other things) the constitutional convention with a legitimacy it otherwise would have lacked.
The fact that diversity has always mattered, and only became controversial when the topic was racial (and to a lesser extent, sexual) diversity, unfortunately buttresses Megan McArdle's intuition (via) that much of the contemporary criticism of the ideal stems from "the terrible, pervasive fear that some brown person, somewhere, is getting away with something."