EzRKNN-LLM 1.2 Rockchip RK3588 and Cosmotop NPU Monitoring

Someone mentioned in the comments of one of my videos to have a look at Cosmotop for NPU monitoring of the Rockchip RK3588.

I found this post on Reddit abut the release of Cosmotop v0.3.0.
  / cosmotop_v030_adds_monitoring_support_for_...  
https://github.com/bjia56/cosmotop/re...

Set the executable flag: chmod +x cosmotop
I had to run it with sudo to see the the NPU monitor.
sudo ./cosmotop.

You can use EzRKNN-LLM to run LLMs with the NPU: git clone https://github.com/Pelochus/ezrknn-llm

Install cmake (if not installed): sudo apt install cmake

cd ezrknn-llm && sudo bash install.sh

You can find converted models on HuggingFace by searching for RK3588.

Example: https://huggingface.co/whaoyang/unslo...

Example command: rkllm gemma3-4b-rk3588-w8a8_g128-opt-0-hybrid-ratio-0.25.rkllm 16384 16384

00:00 Intro
00:17 Cosmotop Monitor
03:23 ezrknpu ezrknn-llm
04:04 Installation
05:27 Converted Gemma 3 4B
06:14 Running ezrknpu ezrknn-llm
07:47 Prompt Execution
09:31 Closing Thoughts

If this video was helpful, please like, comment and subscribe!

Bluesky: https://bsky.app/profile/livinglinux....

#Radxa #Rock5 #rockchip #RK3588 #ai #armbian