Coral and Jetson are not very powerful actually, you can't run even medium model in realtime. https://github.com/openai/whisper/discussions/417

And there is latency issue too, you won't get response very fast. If you look for <0.5s latency or the answer, you need to try something like Vosk probably.