2025. 6. 5.
Technical Requirements for On-device AI
On-device AI has the advantage of not incurring server costs, not being dependent on the network, and not having to worry about data leakage because model inference is performed inside the device.
However, the fact that model inference is performed inside the device means that a lot of optimization must be done. High-performance models must be optimized, limited computing resources must be utilized to the maximum, and overhead must be reduced.
Several techniques are used to achieve this. The most important part is to write core logic using C++, which can bring great benefits in terms of memory management and speed. In addition, most AI model runtimes are written in C++, which has the advantage of being easy to utilize.
ZETIC.MLange uses a technology called JNI to connect C++ and Java (Kotlin) codes to perform on-device AI model inference on the Android OS.
What is JNI(Java Native Interface)?
Java Native Interface (JNI) serves as a bridge between Java and native code (C/C++). It defines a comprehensive framework for interaction between these different programming environments:
JNI establishes function calling conventions between the Java VM (Dalvik/ART) and native code
It enables creation and manipulation of Java objects from native code
Handles data conversion (Marshalling & Unmarshalling) between Java and Native environments
Provides native thread management capabilities
Utilizes the JNIEnv structure as the primary interface for native operations
How Can I Use It?
1. Build Native Libraries with CMake
CMake is a powerful build system generator that offers several advantages:
Automatic generation of build files
Support for multiple platforms including Makefile, Visual Studio Solution, and Xcode
Seamless integration with Gradle
You can generate library file (.so) with add_library keyword.
When working with Android, it's crucial to build for all supported ABIs: arm64-v8a, armeabi-v7a, x86, x86_64
2. Link Native Libraries with NDK (Native Development Kit)
The Android NDK (Native Development Kit) is essential for native code development on Android. It provides:
A comprehensive toolkit for using C/C++ code on Android
Tools for building and executing native code
JNI integration for Java/Kotlin interoperability
Independent memory management from Java's garbage collection
Load Library
Libraries linked using NDK can be referenced at runtime using the System.loadLibrary() function.

3. Native Bridging with JNI
In Kotlin, if you use the external keyword and follow the naming conventions below in C++, JNI will finally bridge the code between Kotlin and C++.
Architecture Proposal
Our recommended architecture implements a carefully designed layered approach that separates different concerns: Domain layer interfaces with the application logic, Wrapper layer provides abstraction and type safety, JNI layer handles the Java-Native communication, and Native layer contains the core C++ implementation. This layered architecture: Domain ↔ Wrapper ↔ JNI ↔ Native not only ensures a clean separation of responsibilities but also promotes maintainable, testable code that can evolve independently at each layer. The structure makes it easier to debug issues, modify implementations, and extend functionality without affecting other layers.
ZETIC.MLange's JNI Usage
In our implementation, we've developed a sophisticated approach to handling multiple machine learning frameworks and creating a unified interface. Our primary focus areas are:
Building comprehensive support for multiple ML frameworks including ONNX, QNN, and TFLite, ensuring seamless integration and optimal performance across different model formats and runtime environments
Creating a robust and flexible unified interface through the abstract ZeticMLangeTargetModel class, which provides a consistent API layer while encapsulating the complexity of different ML framework implementations
Pro Tips: More Optimization
To achieve optimal performance, consider these advanced techniques:
Choose between NewObject and NewGlobalRef based on object lifecycle requirements
Implement object pooling to reduce object creation overhead
Minimize marshalling operations to reduce performance overhead
By following these guidelines and best practices, you can effectively utilize JNI in your Android applications while maintaining high performance and code quality.