I think the important thing to remember is that context is everything. In the experience you mentioned, that guy seemed like a jerk. But I am for freedom of speech. Saying a word is off limits for one race is ridiculous (and racist), but to say it should be off limits for all races is flat out wrong. People shouldn't have a particular word banned from their vernacular. Some people use **** in a playful way, others use it to get under your skin. The thing that you must remember is that context is everything. Without context, words are meaningless. If someone just says **** randomly, it shouldn't matter to anyone. Now if someone insults another person by calling them ****, well that's different because the context is different.
The way you used **** in that experience is perfectly fine, because you were not insulting anyone. But even if you were insulting someone, no one should try to censor you, because free speech is having the freedom to say whatever you want, no matter how offensive it may be to some. The thing you have to understand is that someone will ALWAYS be offended by what you and others say. They have every right to be offended and voice their feelings, but they have no right to try and censor you.
I agree that people that constantly use the word **** sound moronic, but people need to stop attempting to censor others. There shouldn't be any limits to speech, just like there shouldn't be any limits to comedy. A better way to explain my position on "offensive words" would be that I recommend you look up a Louis CK or George Carlin video on Youtube on this very word. They explain my feelings on "offensive words" perfectly.