It's not the cultural norm, so if you ditch an interview, you had best go out of your way to be extremely polite about it. You might be better off just slogging it out to avoid burning bridges.