Conservatives believe that GOVERNMENT decision-makers should always or almost always be color-blind in the sense that government should not make decisions that turn on race. Liberals believe that INDIVIDUALS should be color-blind but see ubiquitous evidence that they are not. In deciding where to live, with whom to socialize and all sorts of other matters, individuals make decisions based on race. The conservatives don't deny this, but they call the phenomenon "societal discrimination" or "voluntary segregation," and conclude that the government cannot take any race-conscious measures to remedy it. Liberals, by contrast, see the harms that result from numerous individual private race-based decisions as worse than the harms that result from race-based decisions by the government to remedy these private decisions. Meanwhile, the conservatives deny that such race-based government action is a "remedy" at all, because they deny that private race-based decision making is (at least constitutionally speaking) a harm.
Quickly adding the caveat that I'm not sure whether I want individuals to be "color blind" per se (my ideal world prefers that we can acknowledge difference without fear or degradation, rather than collapse as many "irrelevant" differences into each other so they have no meaning), I think there is a lot to this, and it raises an interesting to the call for color-blindness.
If Dorf is correct, then the color-blind ideology is necessarily a trade-off. We can make the government completely color-blind, thus effectively conceding a color-conscious "private" sphere. Or we can seek to make society more color-blind, but admit a color-conscious government to do it. Either way, we're conceding the existence of color-consciousness somewhere along the line. The only question is whether it is worse in the government or in the private sector.
When they are feeling particularly honest, conservatives have admitted this to be the trade-off. In his concurrence to the recent school desegregation cases, Justice Thomas stated outright that when "private" decisions like voluntary housing patterns cause "racial imbalance," there is no harm and no foul. And when introducing the "color-blind" principle itself in Plessy v. Ferguson, Justice Harlan assured his readers that:
The white race deems itself to be the dominant race in this country. And so it is, in prestige, in achievements, in education, in wealth, and in power. So, I doubt not, it will continue to be for all time, if it remains true to its great heritage, and holds fast to the principles of constitutional liberty.
For the most part, governmental color-blindness has never been argued to stand in opposition to a society that still breaks upon racially defined lines.
Of course, conservatives could argue that without government reifying the existence of racial categories and their salience, color-conscious will wither away in the private sphere as well. The problem with this claim is that, for starters, it seems to be empirically denied by the French experience. More importantly, it's not clear how well that describes even the American experience. As Michael J. Klarman noted in his definitive work on the Jim Crow era:
Most Jim Crow laws merely described white supremacy; they did not produce it. Legal disfranchisement measures and de jure railroad segregation played relatively minor roles in disfranchising and segregating southern blacks. Entrenched social mores, reinforced by economic power and the threat and reality of physical violence, were primarily responsible for bolstering the South’s racial hierarchy. Legal instantiation of these norms was often more symbolic than functional. Thus, more favorable Court rulings, even if enforceable, would not have appreciably alleviated the oppression of southern blacks. [Michael J. Klarman, From Jim Crow to Civil Rights: The Supreme Court and the Struggle for Racial Equality (New York: Oxford UP, 2004), 59-60]
So we get back to the original question. Once again, I believe that anyone seriously defending governmental color-blindness has to concede that, for the foreseeable future, social discrimination will remain present, salient, and extremely damaging to vulnerable classes in American society with little possibility of remedy. The people remain color-conscious--and more importantly, they remain color-conscious in the "classic" sense of privileging certain races and subordinating others. The governmental color-consciousness proposed by liberals would seek to be color-conscious in a progressive sense--attempting to break down those mentalities of privilege and the effects of subordination. Which color-consciousness is more dangerous? That is the question. Because color-blindness has, for the foreseeable future, already sailed.