AN 1011: TinyML Applications in Altera FPGAs Using LiteRT for Microcontrollers
Visible to Intel only — GUID: vna1740365102920
Ixiasoft
4.2.4. Building the LiteRT TinyML Application
- Device tree modifications: By default, the Agilex™ 5 HPS device tree RAM size is 8 MBytes, which is insufficient for a common tinyML application. Increase the RAM size to 1024 MBytes.
- Navigate to the device tree file intel_socfpga_agilex5.dtsi located in the following path.
<install_Dir>/zephyrproject/zephyr/dts/arm64/intel/
- Modify the RAM size
mem0: memory@80100000 { device_type = "memory"; reg = <0x80100000 DT_SIZE_M(1024)>;
- Navigate to the device tree file intel_socfpga_agilex5.dtsi located in the following path.
- Peripheral integration: The device tree file selects the UART controller as the standard UART device by default. Keep the default UART device setup.
- Enable LiteRT for Microcontrollers static library: To enable the static library within the Zephyr project, apply the following commands:
$ west config manifest.project-filter -- +tflite-micro $ west config manifest.group-filter -- +optional $ west update
- Use the generated LiteRT for Microcontrollers static library: Replace the library files downloaded by Zephyr OS environment with the ones created in Building LiteRT for Microcontrollers Static Library.
- Navigate to the location of the downloaded LiteRT library files by Zephyr OS environment.
<Install_Dir>/zephyrproject/optional/modules/lib/tflite-micro
- Replace the following directories with its files:
- signal
- tensorflow
- third_party
- Navigate to the location of the downloaded LiteRT library files by Zephyr OS environment.
- Customize timing functions: The Agilex™ 5 HPS features a 64-bit Arm Cortex-A architecture with 64-bit timers. In contrast, the LiteRT for Microcontroller library is based on a 32-bit platform with a 32-bit timer. To prevent integer overflow, enhance the LiteRT for Microcontroller timing functions to read a 64-bit input from Arm’s 64-bit timer. Navigate to <zephyrproject_dir>/optional/modules/lib/tflite-micro/tensorflow/lite/micro. Replace the following timing function:
- In micro_time.cc:
#define CLOCK_TICKS_PER_SEC 400000000 … uint32_t ticks_per_second() { return (uint32_t)(CLOCK_TICKS_PER_SEC); } … uint64_t GetCurrentTimeTicks() { return (uint64_t)(arch_k_cycle_get_64());}
- In micro_time.h:
uint64_t GetCurrentTimeTicks(); … inline uint64_t TicksToMs(int64_t ticks) { return static_cast<uint64_t>(1000.0f *static_cast<float>(ticks) / static_cast<float>(ticks_per_second())); }
- In micro_profiler.h, you must change all the 32-bit tick variables to 64-bit size.
uint64_t start_ticks_[kMaxEvents]; uint64_t end_ticks_[kMaxEvents]; … struct TicksPerTag { const char* tag; uint64_t ticks; };
In micro_profiler.cc, any variable that applies the above 32-bit ticks variables must be changed to a 64-bit size. With that, a newly modified TFLite library and Zephyr OS are compatible with the Agilex™ 5 HPS processor.
- In micro_time.cc:
- Create the Application Project: Inside Zephyr directory, navigate to:
<Install_Dir>/zephyrproject/zephyr/samples/modules/tflite-micro
Create a new directory tinyml_mnist. Then, create the following files inside the tinyml_mnist main directory:- CMakeLists.txt
cmake_minimum_required(VERSION 3.20.0) find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE}) project(tensorflow_hello_world) set(NO_THREADSAFE_STATICS $<TARGET_PROPERTY:compiler-cpp,no_threadsafe_statics>) zephyr_compile_options($<$<COMPILE_LANGUAGE:CXX>:${NO_THREADSAFE_STATICS}>) target_sources(app PRIVATE src/main.cc src/model/model_data.cc)
- sample.yaml
sample: description: LiteRT app sample name: litert hps common: tags: tensorflow modules: - tflite-micro
- prj.conf
CONFIG_CPP=y CONFIG_STD_CPP17=y CONFIG_REQUIRES_FULL_LIBC=y CONFIG_POSIX_API=y CONFIG_TENSORFLOW_LITE_MICRO=y CONFIG_STACK_USAGE=y CONFIG_DEBUG=y CONFIG_MAIN_STACK_SIZE=262144 CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=262144 CONFIG_HEAP_MEM_POOL_SIZE=262144 CONFIG_THREAD_MONITOR=y CONFIG_THREAD_STACK_INFO=y CONFIG_THREAD_NAME=y CONFIG_TIMER_READS_ITS_FREQUENCY_AT_RUNTIME=y CONFIG_SYS_CLOCK_HW_CYCLES_PER_SEC=400000000 CONFIG_SYS_CLOCK_TICKS_PER_SEC=400000000 CONFIG_ARM_ARCH_TIMER=y CONFIG_TIMING_FUNCTIONS=y
Note that the stack and heap size are being increased to adapt the requirements to run LiteRT application using CONFIG_MAIN_STACK_SIZE, CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE, and CONFIG_HEAP_MEM_POOL_SIZE.
- CMakeLists.txt
- Toolchain setup, compiler flags, and linker setup: This build uses the default settings of a Zephyr project. The generated application has -O0 settings to enable debugging. Compiling a release version can be enabled using CONFIG_SPEED_OPTIMIZATIONS for the -O2 compiler setting or CONFIG_SIZE_OPTIMIZATIONS for -Os in prj.conf file.
- Build the application: Inside the tinyml_mnist directory, create two directories named image and model, and upload the main.cc (Example in Appendix). In image folder, store the MNIST C array samples, and in model folder, store the model_data.cc, model_data.h and model_settings.h source codes generated in chapter 2.
You can start building the application using the following command:
-
$ west build -b intel_socfpga_agilex5_socdk samples/modules/tflite- micro/tinyml_mnist/ -d agilex5_mnist -p
Where -b refers to the board used for this build, and -d is used to create the build directory with all the generated files. Inside that directory, there is a sub-directory named zephyr, where the zephyr.bin and zephyr.elf files reside.
-