Building an on-device medical app for a hackathon. Deadline in 6 hours.

What works:

  • Gemma 4 E2B (Q4_K_M) + mmproj runs on my laptop via llama-server — reads prescriptions fine
  • A text-only SLM runs on my Snapdragon 8 Elite Gen 5 phone via llama.aichat
  • libggml-htp-v81.so runs on the Hexagon NPU via adb shell (logcat confirms nsp1000)

What I need:

  1. Can Gemma 4 E2B with vision run inside an Android APK via llama.cpp mtmd? Or is that a dead end?
  2. Is ExecuTorch QNN the only way to get NPU from an app (not just adb shell)?
  3. Any working GitHub example of multimodal LLM in an Android app?

Fallback plan: Use ML Kit Text Recognition for OCR if Gemma isn't feasible.

Repo: github.com/sohail9972/IQOO_Hackthon_2026

Any pointers appreciated. Thanks!!!

submitted by /u/Different_Insect5627
[link] [comments]