Capstone Kickoff: Connecting Windows, Ubuntu, and RV1126 + Verifying the RKNN Runtime

TL;DR

I’m building an eye-image clarity assessment system and deploying it on Rockchip RV1126 (NPU-enabled). This kickoff milestone focuses on engineering fundamentals: multi-host connectivity, board access, and RKNN runtime verification on the target device. Result: the RV1126 can run RKNN inference successfully with stable performance (~28–31 ms avg, ~32–36 FPS in the provided demo).

1. Project Goal and Engineering Roadmap

The core goal of this capstone is eye image clarity classification/scoring on an embedded NPU platform, with a fair comparison against traditional focus/clarity metrics.

Planned pipeline:

1. Train a model on PC (Python)
2. Export to ONNX
3. Convert ONNX → RKNN (.rknn) for RV1126
4. Deploy to the board and run inference (final form: C/C++ app)
5. Compare with traditional clarity algorithms (accuracy + latency)
6. Add a GUI layer (Qt) without coupling it to the inference core

The key principle: decouple inference core from UI via a stable API (e.g., JSON/file/socket), so adding Qt later doesn’t break the model pipeline.

2. Three-Node Connectivity: Windows ↔ Ubuntu VM ↔ RV1126

To keep the toolchain reproducible and iteration fast, I set up a “three-node” workflow:

• Windows11: daily driver (terminal, file management, SSH clients)
• Ubuntu 20.04 VM (VMware): RKNN toolkit environment (Python 3.8, conversion scripts)
• RV1126 board: target device (Buildroot-based system, NPU runtime)

Milestone achieved: both Windows and Ubuntu VM can reach the board via SSH, and file transfer (SCP/SFTP) is ready for iterative deployment.

2.1 Target OS Fingerprint: Buildroot on ARMv7

The RV1126 system is not a standard Ubuntu/Debian distro; it’s a typical embedded Buildroot image.

How to quickly check out the system:

uname -a
cat /etc/os-release
uname -m

uname -a
cat /etc/os-release
uname -m

Observed environment (high-level):

• Architecture: armv7l
• OS: Buildroot (2018.02-rc3)
• Kernel: 4.19.x

This matters because many common Linux utilities may be missing, so debugging/deployment should follow embedded practices.

2.2 Remote Access: Dropbear SSH

The board does not ship with OpenSSH (sshd) which means we cannot connect it via ssh directly without any action, but it does include Dropbear.

Verification:

command -v dropbear || echo no-dropbear
ls -l /etc/init.d | grep -i dropbear

command -v dropbear || echo no-dropbear
ls -l /etc/init.d | grep -i dropbear

With Dropbear enabled, the board is accessible remotely as long as:

• the board’s IP is reachable
• the SSH port is known (e.g., 22 or a custom port like 2222)
• valid username/password (or keys) are available

Note: on campus networks, IPs may change frequently; SSH still works—you just need to rediscover the current IP.

3. NPU Stack Confirmation: galcore + rknn_server Architecture

On this platform, RKNN inference is typically served through a running service.

Key signals found on-board:

• NPU module loading via init script (e.g., galcore.ko)
• rknn_server process used as a service backend
• runtime libraries present (e.g., librknn_runtime.so, librknn_api.so)
• Rockchip test binaries available under /rockchip_test/npu

This confirms the target is ready for deploying custom .rknn models.

4. Runtime Verification: rknn_inference Demo + Performance

A demo script (rknn_demo.sh) referenced a missing binary (rknn_demo), so I switched to the actually available tool:

• Binary: /rockchip_test/npu/rknn_inference
• Model: /rockchip_test/npu/vgg_16_maxpool/vgg_16.rknn
• Image: /rockchip_test/npu/vgg_16_maxpool/goldfish_224x224.jpg

/rockchip_test/npu/rknn_inference \
  /rockchip_test/npu/vgg_16_maxpool/vgg_16.rknn \
  /rockchip_test/npu/vgg_16_maxpool/goldfish_224x224.jpg \
  1 ###camera(can be 0)

/rockchip_test/npu/rknn_inference \
  /rockchip_test/npu/vgg_16_maxpool/vgg_16.rknn \
  /rockchip_test/npu/vgg_16_maxpool/goldfish_224x224.jpg \
  1 ###camera(can be 0)

Key output confirms the correct runtime/driver stack and successful inference, including perf:

• RKNN runtime: librknn_runtime 1.6.0 (base: 1126)
• Avg latency:
  • ~28.00 ms (without input/output)
  • ~31.00 ms (with input/output)

This is the final proof for this milestone: the board can run RKNN inference reliably.

Finally:what’s been done and what’s to be done in the future

What This Milestone Proves

This kickoff milestone establishes the engineering foundation:

• Stable access to the board (SSH)
• Three-node workflow ready (Windows + Ubuntu VM + RV1126)
• NPU inference stack confirmed (runtime/driver working)
• Performance baseline captured (ms / FPS)

From here, all future progress is “just” model/data/engineering iteration.

In the future

Immediate next milestones:

1. Build an eye-image dataset strategy (capture + synthetic degradations)
2. Define label scheme (2-class / 3-class / score regression)
3. Train a lightweight embedded-friendly model (MobileNet/ShuffleNet class)
4. Export ONNX and convert to RKNN (prefer INT8 quantization with calibration set)
5. Replace demo model with my own .rknn and validate on-board
6. Benchmark vs traditional clarity metrics (fair evaluation)
7. (Bonus) Add a GUI layer (Qt) via a decoupled API

Additonally, all the standard and detailed process are synced on my git:<CashStolen/AI-embedded-system>

And just like before, have a nice day!