Simple EdgeAI apps on the Beagley-AI

BeagleY-AI

As we had limited success with the oficial BeagleY-AI image, we now try the official TI image https://dr-download.ti.com/software-development/software-development-kit-sdk/MD-NQjfZVt1aJ/10.00.00.08/tisdk-edgeai-image-j722s-evm.wic.xz

Here are the repo that we need for these changes:

https://gitlab.com/bby-ai/edgeai-sdk-changes

Here is the Video by Oleg on this topic. He is also the author of these changes.

You will need to install the required toolchain from TI. You will need to install the processor SDK (PROCESSOR-SDK-LINUX-AM67) and PROCESSOR-SDK-RTOS-J722S

Here is some more reading regarding this step,

https://medium.com/@olof-astrand/building-u-boot-and-the-linux-kernel-for-the-beagley-ai-board-0942565ab28b

Learning about device trees

If you do not know about device trees. Watch this, https://www.youtube.com/watch?v=P9tTS46sjYo

Changes made to the original image

As this image was built by TI for the ti 8-GB AM67A and we use the beagleY-AI with only 4GB of RAM, some changes must be done. Here are what Oleh has done:

In TI’s PSDK, there is an emphasis on using what they call “high memory” — memory located beyond the 32-bit addressing range. While it’s not strictly necessary, TI chose to implement it this way. For example, in the TI PSDK, high_ddr_shared_memory starts at the address 0x9_0000_0000, which does not exist on the BeagleY-AI.

In the changes, A memory layout similar to that of J721E boards, such as the SK-J721E and the BeagleBone AI-64. Specifically high_ddr_shared_memory is replaced with ddr_shared_memory, assigning it the physical address 0xc000_0000. These changes are implemented in the Python script gen_linker_mem_map.py, which generates the device tree .dtsi file (used as a template for the Linux device tree), the .h memory layout, and the .cmd linker scripts for all DSPs and CPUs.

Another aspect of the changes involves memory access permissions. These are handled differently compared to other TDA4 PSDK releases (for example, the J721E). In this system, all memory regions are defined in .syscfg files. I had to add one additional region for MCU2_0—it was missing for some reason, and its absence caused crashes in certain cache operations.

https://gitlab.com/bby-ai/edgeai-sdk-changes

We follow the instructions there, but the patching boils down to something similar to this:

Note that Oleh has changed the layout in the repo slightly so this might also not be 100% correct anymore.
cd $HOME/ti/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/
cd board-support/ti-linux-kernel-6.6.32+git-ti/arch/arm64/boot/dts/ti
cp $HOME/sdk-changes/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/linux-dts/* .
cd $HOME/ti/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/board-support/ti-u-boot-2024.04+git/board/ti/j722s
cp $HOME/ti/sdk-changes/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/u-boot/board_ti_j722s/

cd ~/ti/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/board-support/ti-u-boot-2024.04+git/configs
cp $HOME/ti/sdk-changes/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/u-boot/configs/* .
cd ~/ti/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/board-support/ti-u-boot-2024.04+git/arch/arm/dts
cp  $HOME/sdk-changes/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/u-boot/dts/* .

1. In the ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08 make two dirs: BOOT_PART and ROOTFS_PART
   mkdir BOOT_PART
   mkdir ROOTFS_PART
2. Set environment variables:
   export BOOT_PART=`pwd`/BOOT_PART
   export ROOTFS_PART=`pwd`/ROOTFS_PART
   It means that you are already in the ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08 directory.
3. Make linux dtb
   make linux-dtbs
4 Install linux dtb
   make linux-dtbs_install
5. Make U-Boot
   make u-boot
6. Install U-Boot
   make u-boot_install
7. Copy files from BOOT_PART and ROOTFS_PART to corresponding physical partitions on your SD card.
8. In the ti-processor-sdk-rtos-j722s-evm-10_00_00_05 directory go to the sdk_builer subdir.
   make -j 4 sdk
   make firmware
   copy copy vision_apps_eaik with its content psdk_fw/j722s to the /lib/firmware directory on your SD card.
   This step assumes that you have prepared PSDK as it is described in the official TI documentation.

Here are some diffs compared to the am67a by TI,

diff j722s.env j722s.env.org
9,12d8
< default_device_tree=ti/k3-am67a-beagley-ai.dtb
< findfdt=
<       setenv name_fdt ${default_device_tree};
<       setenv fdtfile ${name_fdt}
24a21,22
>
> rproc_fw_binaries= 0 /lib/firmware/j722s-mcu-r5f0_0-fw 2 /lib/firmware/j722s-main-r5f0_0-fw 3 /lib/firmware/j722s-c71_0-fw 4 /lib/firmware/j722s-c71_1-fw

Most notably are these changes,

vision_apps/platform/j722s/rtos/app_mem_map.h
158,161d157
> /* memory for shared memory buffers in high DDR [ size 512.00 MB ] */
> #define DDR_SHARED_MEM_PHYS_ADDR (0xC0000000u)
> #define DDR_SHARED_MEM_PHYS_SIZE (0x20000000u)
> 
256a253,256
< 
< /* memory for shared memory buffers in high DDR [ size 512.00 MB ] */
< #define DDR_SHARED_MEM_PHYS_ADDR (0x900000000u)
< #define DDR_SHARED_MEM_PHYS_SIZE (0x20000000u)

Also Note,

From my experience trying to compile TIDL. I and Oleh belives that it should communicate with TI OpenVX. TI have a couple of wrappers in between. The PSDK itself that can be compiled in ADAS or EDGEAI mode. With these modifications we use only the later one. C7x is definitely used, but note that C7x in AM67A soc has only 256 bits and j721e has 512.

Here are some logs I got while compiling, not the complete log

Compiling ...

  DTC     arch/arm64/boot/dts/ti/k3-j722s-edgeai-apps.dtbo
arch/arm64/boot/dts/ti/k3-j722s-rtos-memory-map.dtsi:122.69-126.4: Warning (unit_address_vs_reg): /fragment@12/__overlay__/vision-apps-c71_1-memory1b1100000: node has a reg or ranges property, but no unit name
arch/arm64/boot/dts/ti/k3-j722s-rtos-memory-map.dtsi:137.57-140.4: Warning (unit_address_vs_reg): /fragment@12/__overlay__/vision_apps_shared-memories: node has a reg or ranges property, but no unit name
make[1]: Leaving directory '/home/olof/ti/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/board-support/ti-linux-kernel-6.6.32+git-ti'
make[1]: Entering directory '/home/olof/ti/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/board-support/ti-linux-kernel-6.6.32+git-ti'
  DTC     arch/arm64/boot/dts/ti/k3-fpdlink-ov2312-0-2.dtbo
make[1]: Leaving directory '/home/olof/ti/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/board-support/ti-linux-kernel-6.6.32+git-ti'
make[1]: Entering directory '/home/olof/ti/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/board-support/ti-linux-kernel-6.6.32+git-ti'
  DTC     arch/arm64/boot/dts/ti/k3-j722s-vision-apps.dtbo
arch/arm64/boot/dts/ti/k3-j722s-rtos-memory-map.dtsi:122.69-126.4: Warning (unit_address_vs_reg): /fragment@12/__overlay__/vision-apps-c71_1-memory1b1100000: node has a reg or ranges property, but no unit name
arch/arm64/boot/dts/ti/k3-j722s-rtos-memory-map.dtsi:137.57-140.4: Warning (unit_address_vs_reg): /fragment@12/__overlay__/vision_apps_shared-memories: node has a reg or ranges property, but no unit name
make[1]: Leaving directory '/home/olof/ti/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/board-support/ti-linux-kernel-6.6.32+git-ti'
make[1]: Entering directory '/home/olof/ti/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/board-support/ti-linux-kernel-6.6.32+git-ti'
  DTC     arch/arm64/boot/dts/ti/k3-v3link-imx219-0-3.dtbo
make[1]: Leaving directory '/home/olof/ti/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/board-support/ti-linux-kernel-6.6.32+git-ti'
make[1]: Entering directory '/home/olof/ti/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/board-support/ti-linux-kernel-6.6.32+git-ti'
  DTC     arch/arm64/boot/dts/ti/k3-j722s-evm-csi2-ov5640.dtbo
arch/arm64/boot/dts/ti/k3-j722s-evm-csi2-ov5640.dtso:62.8-77.4: Warning (graph_child_address): /fragment@3/__overlay__/ports: graph node has single child node 'port@0', #address-cells/#size-cells are not necessary
make[1]: Leaving directory '/home/olof/ti/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/board-support/ti-linux-kernel-6.6.32+git-ti'
make[1]: Entering directory '/home/olof/ti/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/board-support/ti-linux-kernel-6.6.32+git-ti'
  DTC     arch/arm64/boot/dts/ti/k3-fpdlink-imx390-rcm-1-3.dtbo
make[1]: Leaving directory '/home/olof/ti/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/board-support/ti-linux-kernel-6.6.32+git-ti'
make[1]: Entering directory '/home/olof/ti/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/board-support/ti-linux-kernel-6.6.32+git-ti'
  DTC     arch/arm64/boot/dts/ti/k3-j722s-evm-microtips-mf101hie-panel.dtbo
arch/arm64/boot/dts/ti/k3-j722s-evm-microtips-mf101hie-panel.dtso:106.13-126.3: Warning (graph_child_address): /fragment@6/__overlay__: graph node has single child node 'port@0', #address-cells/#size-cells are not necessary
make[1]: Leaving directory '/home/olof/ti/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/board-support/ti-linux-kernel-6.6.32+git-ti'
make[1]: Entering directory '/home/olof/ti/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/board-support/ti-linux-kernel-6.6.32+git-ti'
  DTC     arch/arm64/boot/dts/ti/k3-am67a-beagley-ai.dtb
arch/arm64/boot/dts/ti/k3-am67a-beagley-ai.dts:144.57-147.4: Warning (unit_address_vs_reg): /reserved-memory/vision_apps_shared-memories: node has a reg or ranges property, but no unit name
arch/arm64/boot/dts/ti/k3-am62p-main.dtsi:1290.21-1293.5: Warning (graph_child_address): /bus@f0000/dss@30220000/ports: graph node has single child node 'port@0', #address-cells/#size-cells are not necessary
  also defined at arch/arm64/boot/dts/ti/k3-am67a-beagley-ai.dts:1271.13-1283.3
make[1]: Leaving directory '/home/olof/ti/ti-processor-sdk-linux-edgeai-j722s-evm-10_00_00_08/board-support/ti-linux-kernel-6.6.32+git-ti'

Copy ~/ti/ti-processor-sdk-rtos-j722s-evm-10_00_00_05/psdk_fw/j722s/vision_apps_evm$ to the /lib/firmware directory on your SD card. This step assumes that you have prepared PSDK as it is described in the official TI documentation.

Starting vision apps on the BeagleY-AI image:

Note that we start the same image,

Also it is important to Edit the UEnv.txt file,

#uenvcmd=if test "$board_name" = "am67-sk"; then ; setenv args_all $args_all systemd.hostname=am67a-sk ; fi

# Setting the right U-Boot environment variables
dorprocboot=1
#name_overlays=ti/k3-j722s-edgeai-apps.dtbo

readelf -a vision_apps/vx_app_rtos_linux_c7x_1.out | grep res
  Entry point address:               0xad200000
  [Nr] Name              Type             Address           Offset
  [29] .resource_table   PROGBITS         00000000ad100000  00001000
  C (compressed), x (unknown), o (OS specific), E (exclude),
   00     .resource_table
root@BeagleBone:/usr/lib/firmware# readelf -a vision_apps/vx_app_rtos_linux_mcu2_0.out | grep res
  Entry point address:               0x0
  [27] .resource_table   PROGBITS        a2100000 002000 00008c 00   A  0   0 4096
  C (compressed), x (unknown), o (OS specific), E (exclude),
   01     .resource_table
  Tag_ABI_align_preserved: 8-byte, except leaf SP
  Tag_ABI_optimization_goals: Aggressive Size
root@BeagleBone:/usr/lib/firmware# file  vision_apps/vx_app_rtos_linux_mcu2_0.out
vision_apps/vx_app_rtos_linux_mcu2_0.out: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, not stripped

[ 1002.435151] remoteproc remoteproc2: powering up 79000000.r5f
[ 1002.436534] remoteproc remoteproc2: Booting fw image j722s-mcu-r5f0_0-fw, size 1084320
[ 1002.437694] remoteproc remoteproc2: bad phdr da 0xa2100000 mem 0x8c
[ 1002.444345] remoteproc remoteproc2: Failed to load program segments: -22
[ 1002.452258] remoteproc remoteproc2: Boot failed: -22
[ 1235.045134] Deferred event dump:00000000: 00000100 00100003 00070001 00820000
[ 1235.045162] Deferred event dump:00000010: 24051642 00000102 00000000 00000000
[ 1235.045169] Deferred event dump:00000020: 00000000 00000000 00000000 00000000
[ 1235.045176] Deferred event dump:00000030: 00000000 00000000 00000000 00000000
[ 1281.367315] remoteproc remoteproc4: powering up 78400000.r5f
[ 1281.368675] remoteproc remoteproc4: Booting fw image j722s-main-r5f0_0-fw, size 1084320
[ 1281.372349] remoteproc remoteproc4: bad phdr da 0xa3ffec00 mem 0x1400
[ 1281.379069] remoteproc remoteproc4: Failed to load program segments: -22
[ 1281.387083] remoteproc remoteproc4: Boot failed: -22

Another boot

[  831.328649] remoteproc remoteproc0: powering up 7e000000.dsp
[  831.344081] remoteproc remoteproc0: Booting fw image j722s-c71_0-fw, size 11821280
[  831.344781] remoteproc remoteproc0: bad phdr da 0xad100000 mem 0x8c
[  831.351153] remoteproc remoteproc0: Failed to load program segments: -22
[  831.360680] remoteproc remoteproc0: Boot failed: -22

[  712.536566] remoteproc remoteproc4: powering up 78400000.r5f
[  712.537956] remoteproc remoteproc4: Booting fw image j722s-main-r5f0_0-fw, size 1084320
[  712.541419] remoteproc remoteproc4: bad phdr da 0xa3ffec00 mem 0x1400
[  712.547962] remoteproc remoteproc4: Failed to load program segments: -22
[  712.555912] remoteproc remoteproc4: Boot failed: -22
[  723.906555] remoteproc remoteproc2: powering up 79000000.r5f
[  723.907970] remoteproc remoteproc2: Booting fw image j722s-mcu-r5f0_0-fw, size 1084320
[  723.909178] remoteproc remoteproc2: bad phdr da 0xa2100000 mem 0x8c
[  723.915523] remoteproc remoteproc2: Failed to load program segments: -22
[  723.923796] remoteproc remoteproc2: Boot failed: -22

But do not worry, this is only what is loaded at boot time. Later the demo will load the correct firmware. The segment .resource_table is probably used by the linux kernel to boot the coprocessors correctly.

Here we go thorough how to build the vision apps. We start with the simplest example.

Vision Apps User Guide: C7x Kernel Application

cd sdk_builder
make vision_apps
make firmware

If you want to deploy your own model, I was not successful with TIDL, instead I recommend trying this. https://github.com/TexasInstruments/edgeai-tensorlab