I'm confident that it largely comes down to this. You have no control over how I read you, whether I read you as irritating or pleasing, good or bad. You have no control over whether I see you as primarily a thing to be read, nor of course do I have any such control over your eyes and mind. When we acknowledge that of course there are many people who cannot tolerate the reality of this we may begin to see how those who obsessively hate what they think political correctness is can see themselves as exclusively honest while preaching the wildest delusions.
Actual common sense should tell us that there never has been and never could be a dominant social consensus on what is or is not offensive. The reasons for why anyone could think there has been or naturally should be are maybe too complex for our purposes here. But I do think it's valuable to recall that the mental effects of surburban and rural segregation in the 20th century will linger for a long time. It has encouraged white people raised in such environments to form an overly smooth and narrow concept of what the "mainstream" is; an exaggerated idea of how much commonality in culture thought and habit the American people have ever had. We should also recall that an age of far fewer media outlets more tightly controlled by particular sorts of white men is still well within living memory. The town newspaper financed by local economic pillars with its insistently heroic and existentially central view of private enterprise. three TV channels conveying much the same white skinned white-collarish sensibility. The lemming-like increase in media that began in the late 20th century and continues has closely aligned with historically oppressed groups working to kick the door down and gain some kind of public voice. If we put all of it together we might see where conservative whites are coming from in perceiving an unprecedented negative pressure upon their positive image and peace of mind. There is even something like a bit of truth to their perception.
Or if you're old enough you may remember a book from the early nineties called "All I really Need to Know I Learned in Kindergarten." I recall that in the late nineties on the old dial-tone internet there were Hotmail groups attributing various right-wing bumper sticker quotes to the book that are not actually in it. still the category of people who were most attracted to this book is telling. There is something quite dark at play on the one hand; a generically conservative idea that the only people one need ever rightfully worry about not offending are Authorities, Social Superiors, Parents, Teachers or adult equivalents thereof. And this does put a sense of insult at suggestions that offending minority members is bad in grim light. My main point however is that there is part of us that wants to believe that the manners we need to mind in order to avoid being negatively thought of; (so that we can KNOW that we are not being negatively thought of) to be permanently settled things; though of course this could only be possible in a Fantasyland of one immortal generation.
What many people mean when they say they want "common sense" is a common sensibility whose goodness everyone has the same motive to presume out of hand and thus presume each other's goodness out of hand. Which is to say that what they want; what par of all of us wants, is a sort of magic veto power over the minds of others; a desire to believe that it is only ever not evil to judge us as wrong when we ourselves are intentionally wrong by our own estimation; and who of course is ever such a thing?
There is ultimately no getting around the problem of backlash. Privileged people who feel personally accused by the calling out of old bigotries and unjust hierarchies will intuitively defend themselves with an intensified form of these very things for a time. Still the problems must of course be named aloud at some point if they are ever to be gotten rid of. It is one of those life pains that need to be suffered and gotten over with.
Sarah Silverman tweeted something like the old saw of "deep down everyone wants just wants to be loved" and though it is an old saw it is still a key truth. In looking at the rise of right-wing populism both here and abroad, these fantasies of an impossible degree of unity in thought belief and identity within societies of millions being "natural" I would say that what the Trump supporter wants, what the MRA the fundamentalist or the white nationalist all want, is an environment where they are only ever judged with the same advantage of familiar affection that one finds in families friend groups and sexual relationships. Since political power can never give them that they shall of course remain dissatisfied. And rather then accept that nothing else at all can ever give them that they will choose instead to contimue blaming dark conspiratorial forces; (Hollywood, bureaucrats, elitists, hipsters, feminists, on and on and on) for denying them this hallucinatory birthright.