Native Americans & Blacks shared a commonality, being to weak, to few, with no political or social influence, to make a difference. Put all that together, and it didn't matter who they supported.....it just didn't matter in the big picture.
Same was true during the Civil War, although some all black units distinguished themselves in battle wearing blue and grey.
Slavery was alive and well, well before the 1600's. Slaves a world wide commodity since the dawn of history.