Will google penalize you for linking to non related sites?

[vero]

I've heard different opinions on this subject. I swap links with any website as long as I feel my type of customer visits those sites and if it will bring me business. Am I wrong for thinking this way? Will google penalize me in the rankings for some of these non related sites?
_________________
outre hair dark poetry