People can automatically be irritated by aloofness. It can easily look like a rejection. People tend to exist between two polls of aloofness, and the balanced middle ground. That's three positions then.
1) I'll call "top aloof" this: it's the vulnerable person who wants to hold onto their pride. They can't imagine life without it, and although it's isolating them, they do feel above others.
2) Whilst "bottom aloof" is another vulnerable position, and often isolating too, but it stems more from low confidence, and/or low self-esteem.
Outsiders can mix these two up, especially in work situations. "Top aloof" don't want to seem arrogant to colleagues, because it's distasteful, whilst "bottom aloof" fear, in terms of their career, if they're known to lack confidence.
3) I've often repeated the words, "balance, respect, and love". They apply here, this third position being the "balance" or middle ground, suggesting nothing is totally black or white. We're all somewhere between top and bottom aloof.
The twist to this follows. We should not assume people are a "top aloof" or a "bottom aloof", because our annoyances of someone's standoffishness, may result in their cruel treatment if we get it wrong. Yes, a little humiliation is good for us all occasionally, but humiliating those on their "hands and knees", can lead to psychiatry or worse. I've mentioned the "respect" aspect often, in terms of individualism, it's important. As for love; yes, but what's your definition of it?