That objection doesn't make any sense--you can't simply use other words when the specific topic is filler words!
I think the objections originally came from some black students. I also read that some Chinese students (or some members of the Chinese community at least) are objecting to the objecting, on the basis that it's disrespectful to their language if it can't be used.
Innocent in one language can be naughty in another. Long ago my wife was puzzled when I (to her ears) told her that what she was looking for was at the back of the cunt section. When talking to her I do often use Chinese words that I know mixed into English so a mixed-language sentence is nothing unusual and she didn't understand what I was actually saying so she interpreted it in Shanghainese (it wasn't a word in Mandarin) and was puzzled.
	
		