Paul makes it clear on Romans 14 and the above Colossians verses that there is nothing wrong in abserving special days and eating whatever you want as long as you are in clear conscious of NOT doing anything that is an afront to God.
Paul says that these rules were created by Man and impossed on Man and that they have no value (other than pointing out what those that wrote those restrictions want you to do ot not do) because they are based on human commands.
It was one Paul's many jabs are orthodox judaism and their "obssession" with ruels and regualtions that they themselves created and impossed on each other.