[Wait, is that guy saying that he's never fought a trainer battle before, or is he saying that he has never gotten into a fistfight?

...Mayyyybe he should cover both of those options in his response. Just in case]

Well, the police will get angry if you just punch someone in the face without warning, and most people don't actually deserve it all that much. (If they do, aim for the eyes.)
But I've never seen someone from this world actually refuse to participate in a Pokémon battle before. I'm not even sure people have the right to do so.