Rather than throwing them out, please save the stamps that arrive on letters and parcels through your door, and let me have them. Any I don't want I will pass on to my sister who does the stamps for the Oxfam shop in Carmarthen, raising money for charity.
I collect Great Britain (used - see my wants list ) and Israel (mint with tabs) and have lots of swaps if you or a friend are interested.
To get started yourself, I advise specialising, as an all world collection would fill a church. Having a nice album to put them in is pleasing. You can buy a ready made one from Stanley Gibbons, or create your own if you have a printer and wish to save money: visit Ebay and search for 'print your own stamp album'.Or use sheets of paper, blank or maybe lined, such as graph paper.
I only collect used stamps. Older stamps are best left on envelopes (e.g. photo below, a letter sent to my great great grandmother). They may be worth more like that. You can collect envelopes mailed to your parents and grandparents which have interesting stamps. There are albums abailable to hold them - search for first day covers album on Ebay.
Otherwise, float the stamps off their backing paper by placing them in a tray of water for an hour. I put them face down (see photo). They should then peel off happily. (Some recent stamps will not peel off. I let those ones dry and then snip off as much of the backing paper as I can.) I leave the floated stamps on a tea towel to dry, and then put them under a book overnight to flatten them out. I use folded stamp hinges to mount them in my album. These are available cheaply on Ebay - see here for example. Moisten half the sticky side and attach it to the stamp, then after a minute, mosten the other half to stick it onto the page.
I find the website Collect British Stamps useful. Lots of free advice, and no request for donations.
Have fun! Thanks, David