What qualifies a perverted guy for 'dirty old man' status: the age difference between the man and girl, or the age of the man in general?

I was just chatting with a fellow perv, and we were wondering what would qualify a person for 'dirty old man' status: the age difference between the guy and girl, or just the guy's age?

Age differenceAge of pervert