Julie Cowell says the famously caustic music mogul started insulting her when he was just five years old.
The 82-year-old former dancer said: "Simon was a determined and often difficult child. When he was very small I bought a new hat and asked him if I looked nice in it. He replied, You look like a poodle.
Despite this, Julie insists her son is not as mean as he comes across on TV talent shows like 'The X Factor' and 'American Idol'.
She told Britain’s Yours magazine: It would only upset me if he really was a nasty person. Simon isn’t nasty he’s just honest. I’ve always insisted on politeness, so when people say how pleasant Simon is, I mutter thank goodness.
It means more to me than his television career. Away from television Simon is just Simon, the same person he’s always been.
Julie also revealed Simon vowed not to get married when he was a teenager.
She explained: He said at 19 that he wouldn’t get married. I thought he might change his mind, but he’s now 50 and it doesn’t look likely to happen.
Click here for more Entertainment News