Introduction

Face detection is a well-known task in computer vision, and Mediapipe is at the forefront of this technology. In this blog post, we'll explore how to implement Mediapipe face detection on various mobile devices using ZETIC.MLange, a powerful framework for on-device AI applications. After this post you can make your own on-device object detection app utilizing Mobile NPUs.

What is Face Detection

The Face Detection model in Google’s MediaPipe is a high-performance machine learning model designed for real-time face detection in images and video streams.

Face Detection Google AI Document : link

What is ZETIC.MLange?: Bringing AI to Mobile devices

ZETIC.MLange is a On-device AI framework that enables developers to deploy complex AI models, like Face Detection, on mobile devices with target hardware utilizations. It leverages on-device NPU (Neural Processing Unit) capabilities for efficient inference.

Github Repository

We provide Face Detection demo application source code for both Android and iOS. repository

Implementation Guide

0. Prerequisites

Prepare the model and input sample of Face Detection from hugging face.

1. Face Detection model

$ pip install tf2onnx
$ python -m tf2onnx.convert --tflite face_detection_short_range.tflite --output face_detection_short_range.onnx --opset 13

2. ZETIC.MLange module file

• Android: Link

• iOS: Link

Step 1. Generate ZETIC.MLange Model Key

Generate MLange Model Key with mlange_gen

# (1) Get mlange_gen
$ wget <https://github.com/zetic-ai/ZETIC_MLange_document/raw/main/bin/mlange_gen> && chmod 755mlange_gen

# (2) Run mlange_gen for two models
#    - Face detection model
$ ./mlange_gen -m face_detection_short_range.onnx -i

Expected output

Step 2. Implement ZeticMLangeModel with your model key

Anroid (Kotlin):
For the detailed application setup, please follow deploy to Android Studio page

val faceDetectionModel = ZeticMLangeModel(this, 'face_detection_short_range')

faceDetectionModel.run(inputs)

val outputs = faceDetectionModel.outputBuffers

iOS (Swift):
For the detailed application setup, please follow deploy to XCode page

let faceDetectionModel = ZeticMLangeModel('face_detection_short_range')

faceDetectionModel.run(inputs)

let outputs = faceDetectionModel.getOutputDataArray()

Step 3. Prepare Face Detection image feature extractor for Android and iOS

Android (Kotlin)

// (0) Initialize ZeticMLangeFeatureFaceDetection
val feature = ZeticMLangeFeatureFaceDetection()

// (1) Preprocess bitmap and get processed float array
val inputs = feature.preprocess(bitmap)

...

// (2) Postprocess to bitmap
val resultBitmap = feature.postprocess(outputs

iOS (Swift)

import ZeticMLange

// (0) Initialize ZeticMLangeFeatureFaceDetection
let feature = ZeticMLangeFeatureFaceDetection()

// (1) Preprocess UIImage and get processed float array
let inputs = feature.preprocess(image)

...

// (2) Postprocess to UIImage
let resultBitmap = feature.postprocess(&outputs)

Step 4. Putting It All Together

Android (Kotlin):
Face Detection Model

// (0) Initialization Models
val faceDetectionModel = ZeticMLangeModel(this, 'face_detection')

// (1) Initialization Feature
val faceDetectionFeature = ZeticMLangeFeatureFaceDetection()

// (2) Preprocess Image
val faceDetectionInputs = faceDetectionFeature.preprocess(bitmap)

// (3) Process Model
faceDetectionModel.run(faceDetectionInputs)
val faceDetectionOutputs = faceDetectionModel.outputBuffers

// (4) Postprocess model run result
val faceDetectionPostprocessed = faceDetectionFeature.postprocess(faceDetectionOutputs

iOS (Swift):
Face Detection Model

// (0) Initialization Models
let faceDetectionModel = ZeticMLangeModel('face_detection')

// (1) Initialization Feature
let faceDetectionFeature = ZeticMLangeFeatureFaceDetection()

// (2) Preprocess Image
let faceDetectionInputs = faceDetectionFeature.preprocess(bitmap)

// (3) Process Model
faceDetectionModel.run(faceDetectionInputs)
let faceDetectionOutputs = faceDetectionModel.getOutputDataArray()

// (4) Postprocess model run result
let faceDetectionPostprocessed = faceDetectionFeature.postprocess(faceDetectionOutputs)

Conclusion: Face Emotion Recognition and On-Device AI - Innovation at the Edge and Limitless Potential

The integration of face detection with On-Device AI represents a pivotal advancement in the evolution of intelligent systems. With the advent of neural processing units (NPUs) in mobile and edge devices, face detection has become faster, more efficient, and more secure, capable of operating without the need for constant cloud connectivity. This technology offers real-time face analysis that is not only responsive but also respects user privacy by processing data locally on the device.

On-Device AI dramatically reduces latency, enhances security, and lowers costs by eliminating the need for cloud-based processing. This makes it possible for a wide variety of applications, such as security systems, biometric authentication, and personalized user experiences, to operate efficiently in both online and offline environments. The edge-based processing approach ensures that even in scenarios with limited or unreliable network connections, devices can still perform face detection with high accuracy and reliability.

Do you have more questions? We welcome your thoughts and inquiries!

• For More Information: If you need further details, please don't hesitate to reach out through ZETIC.ai's Contact Us.

• Join Our Community: Want to share ideas with other developers? Join our Discord community and feel free to leave your comments!

Your participation can help shape the future of on-device AI. We look forward to meeting you in the exciting world of AI!