LocalAIBenchmark is an Android application designed for tech reviewers, developers, and AI enthusiasts to evaluate On-Device AI (Edge AI) inference performance on smartphones. The app runs controlled Small Language Model (SLM) benchmarks using Google's Gemma 3 1B model via Google MediaPipe GenAI LLM Inference API and reports real-time system metrics (CPU clock speed, CPU usage, RAM utilization, and tokens-per-second throughput).
Due to file size limitations, the model .task binary is NOT included directly in this repository.
- You must manually download the Gemma 3 1B INT4 model file (
gemma3-1b-it-int4.task).- Official source: Kaggle - Gemma MediaPipe Models or Hugging Face.
- Save the file with the exact name:
gemma3-1b-it-int4.task - Place the downloaded
.taskfile into one of the following asset locations in the project:- Internal Assets:
app/src/main/assets/gemma3-1b-it-int4.task - OR Asset Pack:
model_assets/src/main/assets/gemma3-1b-it-int4.task
- Internal Assets:
- On-Device LLM Benchmark: Runs local inference using MediaPipe GenAI and calculates tokens per second (t/s) and generation latency (ms).
- Real-Time Hardware Metrics: Monitors CPU frequency, active CPU core usage, and RAM consumption in real-time with dynamic Compose Line Charts.
- Dynamic Performance Scoring: Evaluates your device hardware & inference throughput to assign a performance score and device class rating (Flagship, Premium Mid-Range, Standard Mid-Range, Entry-Level).
- AI Parameter Controls: Adjust Temperature, Top-P (Nucleus Sampling), Top-K, Max Tokens, and Random Seed dynamically.
- Share Benchmark Results: Generate and share summary benchmark cards directly to other apps.
- Language: Kotlin & Java
- UI Framework: Jetpack Compose with Material 3 Design
- Charts: Compose Charts (
ir.ehsannarmani.compose_charts) - AI Engine: Google MediaPipe GenAI Tasks (
com.google.mediapipe:tasks-genai) - Asset Management: Play Asset Delivery (
com.google.android.play:asset-delivery-ktx) - Monetization: Google Mobile Ads & Android In-App Billing
-
Clone the Repository:
git clone https://github.com/EfeAtesh/LocalAIBenchmark.git cd LocalAIBenchmark -
Add Model Asset:
- Create the directory
app/src/main/assets/if it doesn't exist. - Copy
gemma3-1b-it-int4.taskintoapp/src/main/assets/.
- Create the directory
-
Build via Android Studio or Gradle:
./gradlew assembleDebug
-
Run on a Physical Device:
- For realistic LLM performance and hardware metric readings, running on a physical Android device is recommended.
- Download on Google Play: Local AI Benchmark
- Check out LibreChat for an on-device chat client.
This project is licensed under the MIT License - see the LICENSE_Version3 file for details.