Skip to content

Streamline README #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dav-mac merged 4 commits into from
Oct 7, 2026
Merged

Streamline README #3

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
0eb8b67
streamlined readme
dav-mac Oct 6, 2026
e6eb180
Update README.md
dav-mac Oct 6, 2026
efe6720
Delete docs directory
dav-mac Oct 6, 2026
8da38f2
Merge branch 'main' into streamline-repo
dav-mac Oct 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
168 changes: 37 additions & 131 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,131 +1,37 @@
# Optimize and Deploy AI Inference with ExecuTorch on Arm

[![Repository integrity](https://github.com/ArmDeveloperEcosystem/arm-courses-code-executorch/actions/workflows/repo-integrity.yml/badge.svg?branch=main)](https://github.com/arm-education/executorch_on_arm_course/actions/workflows/repo-integrity.yml)
[![Module 3 Android](https://github.com/ArmDeveloperEcosystem/arm-courses-code-executorch/actions/workflows/module-3-android.yml/badge.svg?branch=main)](https://github.com/arm-education/executorch_on_arm_course/actions/workflows/module-3-android.yml)

This repository accompanies the hands-on course for deploying PyTorch models to Arm CPUs and NPUs with
ExecuTorch available on learn.arm.com, [_Optimize and deploy AI inference with ExecuTorch on Arm_](https://learn.arm.com/courses/executorch-on-arm/)

The course walks through ahead-of-time model preparation,
backend delegation and graph inspection, post-training quantization, and runtime benchmarking and example applications across a range of hardware targets.

You will start with CPU inference on Raspberry Pi using XNNPACK, covering examples including both transformer and CNN-based models. You will then explore Mobile deployment, and understand what changes when moving from Cortex-A on Edge Linux, to a Mobile OS like Android. Finally, you will get hands-on with Cortex-M Microcontroller + Ethos-U NPU acceleration. Each Module should extend your knowledge, and provide practical examples that allow you to quickly begin trying the workflow. Each Module links to supplementary resources that act as useful next steps, either to reinforce what you have done, or to extend to more advanced topics.

This course is best used in parallel with the ExecuTorch Documentation.

## Course content

This repository contains the necessary scripts and information to accompany the course.

The learner-facing Modules 1–4 are published separately at [URL TO FOLLOW].
This repository contains the supporting scripts, applications, images, and
configuration used by those modules.

## Course Overview

**Skill level:** Introductory

**Course length:** 8-10 hours

**Author:** Matt Cossins, with contributions from Marcelo Rovai

**Main technologies:** PyTorch, ExecuTorch, XNNPACK, KleidiAI, Model Explorer, Cortex-A, Cortex-M, Ethos-U, TOSA,
Vela, Corstone-320, Transformers, LLMs, CNNs

**License:** [BSD-3-Clause-Clear](https://spdx.org/licenses/BSD-3-Clause-Clear.html)

## Who This Is For

This course is for developers who want to learn how to deploy PyTorch
models efficiently on Arm-based edge devices, using the ExecuTorch Runtime.

You should be comfortable with basic Python and familiar with core
machine learning concepts such as models, tensors, inference, and
quantization. You do not need prior ExecuTorch experience - this is ideal for beginners with ExecuTorch.

You are encouraged to use AI Agents when taking this course, and skills are provided to aid your understanding, and next steps. See Module 0 for more information.

## Hardware and Environment

The full course uses:

- macOS or Ubuntu, including Ubuntu under WSL, for development-machine steps
- Raspberry Pi 5 running 64-bit Raspberry Pi OS Trixie for Modules 1 and 2
- Android smartphone or Emulation via Android Studio for Module 3
- Ubuntu LTS host or cloud instance for Module 4

If you do not have all target devices, you can still follow the
ahead-of-time preparation scripts and either read the content without running the benchmarks on hardware, or try running them on your local development machine instead.

## Course Modules

### Module 0: Course and ExecuTorch Overview

Introduces ExecuTorch, the ahead-of-time deployment workflow, the course
hardware targets, and the optional AI-agent learning modes.

### Module 1: Get started with ExecuTorch and Understand Delegation - Transformer Inference on Arm Cortex-A CPU

In this Module, we introduce ExecuTorch, explaining its role in the PyTorch Ecosystem, and in Edge AI. You will install ExecuTorch to your development machine in a Python environment. In an `ahead-of-time` flow, you will obtain two different Transformer-based PyTorch models from Hugging Face (GPT-2 and DistilBERT), and perform export and lowering to convert them to an ExecuTorch format. You will use both the ExecuTorch Portable Kernel Library, and also optimized delegation to XNNPACK.

After comparing your original PyTorch models, your ExecuTorch Portable models, and your optimized XNNPACK delegate ExecuTorch models - including via the use of Model Explorer - you will benchmark each on your target device: a Raspberry Pi 5.

Finally, you will run example applications for each: a simple text-generation application and an autocomplete application.

By the end, you will have completed a typical `ahead-of-time` preparation flow, and deployed your prepared models in applications on target hardware. You will be able to discuss when and why to delegate to optimized backends like XNNPACK, and how you can use tools like Model Explorer to visualize graph changes after delegation has occurred. You will be ready to move onto Module 2, where you will cover image classification, and will also explore Quantization in ExecuTorch.

### Module 2: CNN Inference on Arm Cortex-A CPU and Optimize with INT8 Quantization

In this Module, you move from transformer inference to image classification.
You will prepare a pretrained MobileNetV2 model from TorchVision, and export the model to an ExecuTorch `.pte` artifact with XNNPACK delegation. You will benchmark two runtime paths on the Raspberry Pi: PyTorch eager and
FP32 ExecuTorch with XNNPACK.

You will then add post-training quantization
(PTQ), using calibration images to create an INT8 ExecuTorch with XNNPACK
`.pte` artifact. This lets you compare FP32 and INT8 deployment tradeoffs:
artifact size, target-side latency, delegation, and top-5 classification
output. You will again run an example application, a simple image classifier web app.

An optional profiling section builds an ETDump-enabled runtime on the Raspberry
Pi, captures ETDump and ETRecord data, examines it with the ExecuTorch Inspector,
and opens a timing overlay in Model Explorer.

By the end, you will be confident quantizing models with post-training
quantization and explaining the benefits and tradeoffs of an INT8 `.pte`
artifact. You will also understand how CNN model architectures delegate
differently to XNNPACK when compared with transformer-based models, and you
will be able to compare PyTorch eager, FP32 ExecuTorch with XNNPACK, and INT8
ExecuTorch with XNNPACK on the same target device.

### Module 3: ExecuTorch on Mobile - Using the XNNPACK Backend on Android for Semantic Segmentation

In this module, you move from Edge Linux deployment on a Raspberry Pi to an
Android application. You will prepare an LR-ASPP MobileNetV3 semantic
segmentation model, lower an FP32 version with XNNPACK, and validate its object
cutout output on the development machine. You will then apply post-training
quantization, compare the FP32 and INT8 results, and select the INT8 `.pte`
artifact for the mobile application.

You will package the model, labels, and preprocessing settings as Android
assets and run the application on a physical Arm phone or an Android Studio
emulator. The Java application uses the ExecuTorch Android API to load the
model, create input tensors, run `forward()`, interpret the semantic
segmentation logits, and produce an image with a transparent background.

By the end, you will understand what changes when moving an ExecuTorch
application from Python on Edge Linux to Java or Kotlin on Android, including
application assets, native ABIs, lifecycle management, background inference,
and native runtime cleanup. You will also understand where XNNPACK CPU
execution fits alongside other mobile options such as iOS, device NPUs, SME2,
and the Arm VGF backend for neural graphics workloads.

### Module 4: Deploy ExecuTorch Models with Arm Ethos-U Acceleration

In this module, you return to the MobileNetV2 model, and explore how to optimize and accelerate its deployment via use of an Ethos-U NPU. You will explore FP32 and INT8 Post-Training Quantization routes, and understand how the Ethos-U backend differs from the XNNPACK backend. You will become familiar with the Vela compiler and the TOSA intermediate representation.

You then add an unsupported operation to split the model into two delegated
regions and compare the fallback path. After returning to the clean INT8 model,
you run it on a Corstone-320 FVP simulation of the Cortex-M + Ethos-U and inspect
the generated `.pte` graph and FVP output. The module finishes by adapting the
generic C++ runner into a MobileNetV2 classification application that prints
ImageNet labels over UART.
# Course code for 'Optimize and deploy AI inference with ExecuTorch on Arm'

[![Repository integrity](https://github.com/ArmDeveloperEcosystem/arm-courses-code-executorch/actions/workflows/repo-integrity.yml/badge.svg?branch=main)](https://github.com/ArmDeveloperEcosystem/arm-courses-code-executorch/actions/workflows/repo-integrity.yml)
[![Module 3 Android](https://github.com/ArmDeveloperEcosystem/arm-courses-code-executorch/actions/workflows/module-3-android.yml/badge.svg?branch=main)](https://github.com/ArmDeveloperEcosystem/arm-courses-code-executorch/actions/workflows/module-3-android.yml)

This repository contains the supporting scripts and example applications for
[Optimize and deploy AI inference with ExecuTorch on Arm](https://learn.arm.com/courses/executorch-on-arm/).
Use the course for prerequisites, setup, and exercise instructions. The links below
take you to the code for the module you are following.

## Find your module

| Module | Code |
| --- | --- |
| 1 | [Transformer inference on Raspberry Pi](module-1-transformer-inference-with-xnnpack/) |
| 2 | [Image classification and INT8 quantization on Raspberry Pi](module-2-cnn-inference-with-quantization/) |
| 3 | [Semantic segmentation on Android](module-3-executorch-on-android-semantic-segmentation/) |
| 4 | [Ethos-U inference with Corstone-320](module-4-ethos-npu-accelerated-inference/) |

## Using this repository

- Follow your lesson's environment, working-directory, and command instructions.
Requirements differ between modules, so use the versions and installation steps
specified for that exercise.
- Clone and keep the complete repository together. Module scripts import shared
helpers from [course_support](course_support/). These helpers support tasks such
as calibration and benchmarking across the course modules. Copying a single
module directory can leave those imports unavailable.
- Open the script or application named in the lesson to connect each command with
its implementation. Continue using the course to interpret outputs and decide
the next step.

Optional [AI-agent skills](.agents/skills/) support working with the code; Module 0
of the course explains how to use them.

For proposed fixes or improvements, see [Contributing](CONTRIBUTING.md).
Code is provided under the [Clear BSD License](LICENSE).
Loading