Introverts are a much more rare than extroverts, so we often get aimless connotations such as the following: antisocial, very shy, coldhearted, arrogant, etc.
It's as simple as this:
Extroverts derive their energy from other people
Introverts derive energy from within
(shy people are actually extroverts who long for a better relationship/connection with others, but are fearful for whatever given reason)
If an introvert wants to chill out, then s/he spends time reading or listening to music. If an extrovert wants to recharge, then s/he goes to a party or restaurant with friends.
Jungian cognitive functions explains this (as well as other aspects) well. Personally, I'm an INTP. (Introvert, iNtuition, Thinking, Perceiving)
The word "introvert" gets abused, because people often link it to being shy or standoffish.