You will be trading a bit of convenience for your increase in privacy. How many steps you take, and on which occasion you take them will depend on your degree of trust 🙂
1. cookies – by disabling persistent cookies, you can prevent a website (or network of websites) to track your browsing pattern across multiple sessions. Each time you close your browser, any cookies you have received will be deleted. The main drawback is that this will disable any website auto-logins or specific settings you may have enabled.
2. toolbars – assistants, helpers and other gimmicks that you might want to install as a browser toolbar will periodically phone home while you are surfing the web. Some of this may be legit, some of it may not.
3. websites that you register for – Social networking sites or webmail services may be interested in following your activities while you browse their site. Which topics interest you, what opinions did you express, did you make any purchases?
4. geolocation of your ip address – great for targeted advertising, or to display varied content (e.g. languages) depending on a user’s location. You could consider using a proxy that does not relay your original ip address.
5. data harvesting at the ISP level – search for Phorm, Nebuad, Carnivore 😉
The list could go on for a while, many others can comment on the other breeches of security like… Flash cookies, user agents, hopcounts, private browsing, browser anti-phishing features, omnibox, anonymizers, hosts file anyone?