This is incredible work! Porting 11 model families already is a massive feat. Out of curiosity, do you think this could be exposed to the Shortcuts app to trigger local model inference via system automations? Having these run entirely on-device via Shortcuts would open up so many insane workflow possibilities.

Agreed @simonw and if you can give positive examples you get better results with negative examples of what not to do

Thank you for sharing your story! I wish you continued success. Hope you keep building and keep inspiring

hey Prasad out of curiosity what was your local setup. Ollama or MLX or GGUF files? And more importantly how good/complete were the response?