A man who is blind said he was kicked out of a gym for "staring" at a woman while she was working out.