The high court had also ruled in Brown v. Board of Education against school segregation but schools and states had "ignored it with impunity, " Obama recalled.