Why do we have to give back to society?

Every day or so I hear or read about someone demanding that others must give back to society. This makes me wonder what they have stolen. The phrase "giving back" implies that individuals have obtained something that they were not entitled to receive. Why do we demand that people who have paid their taxes and earned their money need to give it back? Do we really believe the wealthy stole everything they have? Please do not misunderstand. I highly value the opportunity to serve others and share my meager, middle class income with those who need it or to help my community become a better place for all to live. In fact, I regularly donate to local, National and international charities, scholarship funds and other causes I value. I am also involved in several non-profit organization on the local and International levels. I pay my "Fair Share" of taxes as specified by my elected leaders. These taxes fund my share of education and the public infrastructure. I resent the hell out of someone demanding that I give my money to serve their purpose because I stole it from society!