Stamps may be purchased at post offices or tobacco shops (tabacchi). You can locate a tabacchi by looking for a hanging white T in a dark, rectangular sign on the outside wall of the store.

Post offices are open:

Monday to Friday from 8:15 am to 2:00 pm
Saturday from 8:15 am to 12:00pm

The main Post Office is located in Piazza San Silvestro and is open:

Monday to Friday from 8:00 am to 9:00 pm
Saturday from 8:00 am to 12:00pm

Mail boxes are spread throughtout the city and are red with 2 slots:
The slot for local (Rome) mail is marked "Per la Citta' ".
The slot for all national and international correspondence is marked "Altre destinazioni".

One can also mail from Vatican City in St. Peter's Square. Mail sent from here is thought to reach its final destination more quickly.
However, this may just be mis-information propogated by Romans to keep the lines at the city post offices shorter.
To our knowledge, it has never been scientifically tested.