I think folks might have the wrong idea about what the "legacy of slavery" (or Jim Crow or discrimination) is. As far as I can tell, at least in the popular White imagination it seems to be some genetic contribution to the Black psyche which haunts them and holds them back from pursuing the opportunities America has made available today. There is a smidgen to that definition -- in that it is psychologically salient to know that your ancestors were treated by this regime as subhuman animals who could kidnapped, sold, beaten, and even killed at will. This doesn't even actually rely on being a descendant of slaves: it works very similarly when you consider that "had I been there, I'd have been enslaved too" (Jews can relate to this same phenomenon -- we know what would have happened to us in the Holocaust or in a Russian pogrom, even if none of our immediate ancestors were survivors of Nazi or Czarist or Soviet tyranny). But I think that there is a lot more to the legacy of slavery that doesn't fit into that at all.
First, racist policies have by and large prevented Blacks from accumulating wealth across generations. Not only is this inherently disadvantaging compared to their peers, but it may develop habits of saving, spending, and consumption which may be beneficial survival skills in a Jim Crows style society, but may not be optimal now.
But the even bigger element which I think is missed is how slavery and Jim Crow helped create a network of social understandings regarding the place, value, and characteristics of Black people which are still operative today. This gets dropped from the conversation because it has to do with how contemporary Whites relate to Blacks and Black history -- which of course we (say we) don't do, because we (say we) are color-blind. But insofar as the image of the Black in the White imagination is an image in part defined by subjugation and inferiority, that legacy is still very much an important part of the Black experience (and again, it really isn't dependent on whether one is the descendant of slaves).
All of which is good reason to study the subject more -- a position which is admittedly easy for me to take given how moronic the arguments trotted out by the opposition are.