Roman society was not ascetic as was the case of Middle Ages, or denial of sexuality as was the case in Victorian era, which witnessed destruction of thousands of arts from Antiquity, Renaissance, and Baroque era deemed to be immoral. According to Joyce E. Salisbury, whose "The Blood of Martyrs: Unintended Consequences of Ancient Violence" mentions that Roman society was deeply conservative, where the family was the backbone of the civilization. Women were subject of their husband or fathers, and they did not have sexual freedom we see it through 21st century eyes.
There are dozens books on Roman family, sex, and scandals from Imperial Rome. Wayne R. Dynes in 1992 characterized Republican Rome as a conservative, authoritarian, and restrictive attitude toward sex, women considered as inferior, and a deep fear of homosexuality. Roman society was militaristic, dominated by mighty army which prohibited sex that would undermine power structure and command line, unlike Hellenistic world, where sexual attitudes were more open.
Generally, in the term of sex, people see Roman society through medieval and early Christian ascetic views, and get their judgment impaired by movies like Caligula. In fact, many people find sexually restrictive, conservative, behavior in Rome shocking.