At their eponymous blog, Richard Posner and Gary Becker offer up a partial defense of legacy preferences in college (particularly elite college) admissions.
Posner's argument is basically that, while legacy preferences are distasteful, their actual harm on society is minuscule -- boiling down to the non-legacy student who is "bumped" going to Michigan instead of Yale. Not only is this simply not a huge problem on its own terms, but whatever marginal harm the student faces by being a Wolverine is counterbalanced by the benefit that other Michigan students get by their association with top-quality students.
Becker is willing to make a more positive case for legacy preferences, observing that college's prefer legacies because they are more likely to accept admission offers, increasing the college's yield rate (he calls it their "harvest", which I think is a far creepier term for the concept) and their rates of alumni giving.
As for me, while I do think it is an inversion of justice that legacy preferences are permissible and racial ones are not, I'm actually more-or-less neutral on the justifiability of legacy preferences on their own terms. And basically, it's my time at Carleton -- a school that, at least in my impression, has a strong legacy presence -- that shifted me from opposition to neutrality.
Legacy admission hasn't directly been a part of my own life -- I attended public school K-12, and to my knowledge I have absolutely zero family connection either to Carleton or the University of Chicago. But, one of the things I really liked about Carleton was the school's sense of institutional memory, and to me, that was intricately bound up in the number of students who had parents, siblings, or other relatives who had attended as well. It created a positive school dynamic and sense of community that really was integral to the school's charm. And I think the alumni body's fierce loyalty to its alma mater is a crucial part of our school's appeal. Carleton has one of the highest giving rates amongst its alumni of any school in the country, and on many occasion, when I've told someone I went to Carleton, they've noted how they've never met a Carleton graduate who didn't gush about their time there.
One of the things I think admissions officers are doing, and should be doing, is trying to construct classes that will together form a vibrant, engaged community. That's one of the reasons I support diversity programs like racial affirmative action -- academic and social communities are simply more robust when they incorporate a broad range of different backgrounds and perspectives. To the extent that legacies help create this sense of vibrancy and community -- and in my experience at Carleton, they did -- that's a valid attribute for admissions officials to consider.
Of course, this does not answer just how much weight legacy status should carry in admissions decisions. Nor does it answer the counter-arguments about "rich-getting-richer" and entrenching inequality. I also think that the normative justifiability of legacy preferences is hinged upon the general permissibility of allowing college's discretion to create diversified academic communities -- it can't be that college's can only look to those non-metric-based personal characteristics when they benefit the already privileged. But, in a world where this sort of holistic admissions process is allowed and is the norm, I think legacy status has a place alongside other personal characteristics as a valid consideration in building the best incoming class possible.