White supremacists have a history of exerting significant power in the west and using it to oppress people. Non-white supremacists have rarely had any significant power to enact their policies.
More over white supremacy was a mainstream political movement for a while, particularly in the 1930s and following decades, but black supremacy has never been so in the developed world.
Even ideologically they differ significantly. White supremacists tend to point to western civilization as proof that white people are superior, and use historical racist ideas about race in their arguments. Black supremacists tend to cite the poor treatment of black people by whites, particularly slavery, to create a moral argument in favour of subjugation and genocide.
In terms of practical actions white supremacists have a lot more "infrastructure" and their politics are at the fringes of powerful mainstream parties like the Republicans in the US and various far right parties in Europe. They have been able to enact policies that create systemic racism, both historically and in the present. Black supremacists have little political power outside of a few African nations.