i was always trained that "Sir" and "Ma'am" was a general, polite, honorific. It does not carry as much weight as, for example, Master.
Even in my vanilla life i use Sir and Ma'am.
For me to say something other than Sir or Ma'am requires it to be earned. i will NEVER call someone Mistress or Master until they've earned and proved it. Furthermore, when Owned i will address a Dominant as my Owner wishes me to.
Sir & Ma'am is simply being polite and respectful in the vanilla and the leather worlds. There is no excuse for rudeness in either.
Anything above and beyond that must be earned. Yes Dominants, you must earn it. i was raised by The Old Guard and i am very opinionated about certain things.