iCreatAI News
Skip to content
AWS Machine Learning Blog· Josh Longenecker·· 25 days agoAI score34

Train Qwen3 on AWS Trainium with torchtitan

Train Qwen3 on AWS Trainium with torchtitan

AI summary

This post walks through training Qwen3 three ways on a single trn2.48xlarge using torchtitan-neuron: via Hugging Face Transformers v5 as a modeling backend, via torchtitan's native Qwen3 by changing config.model_spec, and via NKI mixture-of-experts kernels from the torchtitan-neuron recipes, all using the same torchrun launch pattern. It also shows how to load and save published safetensors checkpoints, attach a custom NKI rope_hf kernel through torchtitan's override registry, and scale with data and tensor parallelism across 64 logical NeuronCores.

Source: AWS Machine Learning Blog · aws.amazon.com