I always thought the point of goosebumps was to warm you up when you're cold, like shivering.
Shivering does warm you up. If somebody is so cold they have stopped shivering they are in trouble. The arectores pilorum muscles are not involved in shivering at all. It is your big skeletal muscles that move rapidly to generate heat.
Do other apes get goosebumps I wonder. Because they are hairy.
Yep that what makes their hair stand on end to trap heat. Exactly the same reaction we have except we have puny hair so it doesn't actually help.