With Great Power Comes Great Control and the freedom to spread aforementioned evil.
Most Chaotic Evil characters don't constantly break the law, but they don't place much value in laws (or, for weaker-CCEs, do not see the value in laws that do not function solely to their depraved objectives or increase their own personal freedom). They believe that their own evil impulses are their best guides, and that tying themselves to any given code of conduct would be limiting themselves. They often react violently to anyone who tries to instill any form of order over them, believing these people to be restricting their freedom. Chaotic Evil characters often focus very strongly on their own individual rights and freedoms, and will strongly resist any form of oppression over themselves.