Introduction

Object detection is a crucial task in computer vision, and YOLOv8 is at the forefront of this technology. In this blog post, we'll explore how to implement YOLOv8 object detection on mobile devices using ZETIC.MLange, a powerful framework for on-device AI applications. After this post you can make your own on-device object detection app utilizing Mobile NPUs.

What is YOLOv8?

YOLOv8 is the latest iteration of the YOLO (You Only Look Once) family of real-time object detection and image segmentation models. Developed by Ultralytics, YOLOv8 offers state-of-the-art performance for various computer vision tasks.

Official YOLOv8 Document page by Ultrylytics: link

What is ZETIC.MLange?: Bringing AI to Mobile devices

ZETIC.MLange is a On-device AI framework that enables developers to deploy complex AI models, like YOLOv8, on mobile devices with target hardware utilizations. It leverages on-device NPU (Neural Processing Unit) capabilities for efficient inference.

Github repository

The result demo application source code for the On-device YOLOv8 demo is prepared at the repository.

Implementation Guide

Prerequisites: Prepare YOLOv8 model and sample data

1. The YOLOv8 model

from ultralytics import YOLO 
import torch 

# Load a YOLOv8 model 
model = YOLO("yolov8n.pt") 

# Export the model 
model.export(format="onnx", opset=12, simplify=True, dynamic=False, imgsz=640)

2. A sample input as a NumPy array

import cv2 
import numpy as np 

def preprocess_image(image_path, target_size=(640, 640)): 
  img = cv2.imread(image_path) 
  img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) 
  img = cv2.resize(img, target_size) 
  img = img.astype(np.float32) / 255.0 img = np.transpose(img, (2, 0, 1)) 
  img = np.expand_dims(img, axis=0) 
  return img

3. ZETIC.MLange module file

• Android: zeticMLangeYoloV8.aar

• iOS: zeticMLange.framework.zip for YOLOv8

Step 1: Generate ZETIC.MLange Model Key

Generate MLange Model Key with mlange_gen

# (1) Get mlange_gen 
$ wget <https://github.com/zetic-ai/ZETIC_MLange_document/raw/main/bin/mlange_gen> && chmod 755 mlange_gen 

# (2) Run mlange_gen 
$ ./mlange_gen -m yolov8n.onnx -i yolo8_detector_input.npy 
  
# Expected output 
# ... 
# MLange Model Key : {**YOUR_YOLOV8_MODEL_KEY**} 
# ...

Step 2: Implement ZETIC.MLangeModel

Android (Java):
For the project setup, please follow deploy to Android Studio page

ZeticMLangeModel zeticMLangeYoloVModel = new ZeticMLangeModel(this, "YOUR_YOLOV8_MODEL_KEY"); 
zeticMLangeYoloVModel.run(inputs); 
ByteBuffer[] outputs = zeticMLangeYoloVModel.getOutputBuffers();

iOS (Swift):
For the project setup, please follow deploy to XCode page

let yoloModel = ZeticMLangeModel(mlange_model_key) 
yoloModel.run(yoloModelInput); 
let outputs = yoloModel.getOutputDataArray()

Step 3: YOLOv8 Image Feature Extractor

Android (Java)

ZeticMLangeFeatureYolov8 zeticMLangeFeatureYolov8 = new ZeticMLangeFeatureYolov8(cocoYamlFilePath); 

// (Preprocess) Get processed float array from Bitmap 
float[] floatInput = zeticMLangeFeatureYolov8.preprocess(bitmap); 

// (Postprocess) Float array to UIImage 
Bitmap resultBitmap = zeticMLangeFeatureYolov8.postprocess(outputFloatArray);

iOS (Swift)

let yoloFeature = ZeticMLangeFeatureYolov8(cocoYamlFileUrl) 

// (Preprocess) Get processed float array from UIImage 
let yoloProcessedData = self.yoloFeature.preprocess(image) 

// (Postprocess) Float array to UIImage 
let yoloResultImage = self.yoloFeature.postprocess(image, &outputs[0])

Step 4: Putting It All Together

Android (Java)

// (0) Initialization 
ZeticMLangeFeatureYolov8 zeticMLangeFeatureYolov8 = new ZeticMLangeFeatureYolov8(cocoYamlFilePath); 
zeticMLangeYoloVModel = new ZeticMLangeModel(this, "YOUR_YOLOV8_MODEL_KEY"); 

// (1) Preprocess image 
float[] floatInput = zeticMLangeFeatureYolov8.preprocess(bitmap); 

// (2) Process YOLOv8 Model 
zeticMLangeYoloVModel.run([floatInput]); 
ByteBuffer[] outputs = zeticMLangeYoloVModel.getOutputBuffers(); 

// (3) Postprocess to bitmap 
Bitmap resultBitmap = zeticMLangeFeatureYolov8.postprocess(outputFloatArray);

iOS (Swift)

// (0) Initialization 
private let yoloModel: ZeticMLangeModel 
private let yoloFeature = ZeticMLangeFeatureYolov8(cocoYamlFileUrl) 

// (1) Preprocess image 
let yoloProcessedData = self.yoloFeature.preprocess(image) 

// (2) Process YOLOv8 Model 
self.yoloModel.run([yoloProcessedData]); 
let outputs = self.yoloModel.getOutputDataArray() 

// (3) Postprocess the output 
let yoloResultImage = self.yoloFeature.postprocess(image, &outputs[0])

Conclusion: YOLOv8 and On-Device AI - Innovation at the Edge and Limitless Potential

The combination of YOLOv8 models and on-device AI is revolutionizing computer vision technology not only on mobile devices but also on edge devices. This powerful synergy offers several key advantages:

1. Realization of Real-time Processing: The fast processing speed of YOLOv8 coupled with the immediate response capability of on-device AI enables real-time object detection and tracking.

2. Enhanced Privacy: As data never leaves the device, the security of sensitive information is significantly strengthened.

3. Offline Functionality: Advanced AI features can be used without an internet connection, ensuring reliable performance in any environment.

4. Energy Efficiency: On-device processing reduces overall energy consumption, extending battery life.

5. Edge Device Optimization: Lightweight versions of YOLOv8 operate efficiently on edge devices with limited computing power. This allows advanced AI functionalities to be implemented even on low-spec devices such as CCTV cameras or IoT sensors.

6. Utilization on Limited Chipsets: YOLOv8 maintains impressive accuracy and speed even on chipsets with limited performance. This enables its use in various fields such as smart home devices, wearables, and industrial sensors.

7. Development of Customized Solutions: Developers can easily create AI solutions tailored to various industries based on YOLOv8.

This combination of technologies opens up endless possibilities. For example:

• Implementation of advanced multi-object tracking systems on low-spec CCTV

• Image segmentation specialized for industrial IoT sensors

• Real-time gesture recognition and emotion analysis on wearable devices

• Providing innovative user experiences combining object recognition and augmented reality on smart home devices

The fusion of YOLOv8 and on-device AI goes beyond mere technological innovation; it's opening new possibilities across our daily lives and various industries. By enabling high-performance AI implementation even in resource-constrained environments, it is leading us into an era of AI-based solutions that are smarter, safer, and more efficient.

Do you have more questions? We welcome your thoughts and inquiries!

• For More Information: If you need further details, please don't hesitate to reach out through ZETIC.ai's Contact Us.

• Join Our Community: Want to share ideas with other developers? Join our Discord community and feel free to leave your comments!

Your participation can help shape the future of on-device AI. We look forward to meeting you in the exciting world of AI!