/gke-ai-troubleshooting-tpu-mxla-hang
用 ML Diagnostics 的 MXLA Hang Analyzer 與 Cloud Monitoring 延遲指標，診斷 GKE Cloud TPU 多 slice 訓練當住（Megascale HANG_DETECTED 日誌）的問題。
- 安裝次數
- 1100
- 專案星數
- 2.2 萬 ★
- 授權
- Apache-2.0
- 專案更新
- 2026-11-02
怎麼裝
npx skills add https://github.com/google/skills --skill gke-ai-troubleshooting-tpu-mxla-hang安裝前請先看過原文內容。中文說明是 AI 翻譯，不保證安全。
原文
Diagnose GKE Cloud TPU multi-slice training hangs (Megascale `HANG_DETECTED` logs and hang monitored events) using the ML Diagnostics `Megascale XLA (MXLA) Hang Analyzer` (`gcloud alpha mldiagnostics monitored-events`) and 1-minute Cloud Monitoring multi-slice latency metrics (`kubernetes.io/container/multislice/*`). Distinguishes XLA compiler/HLO launch divergence and host data-input stalls from TPU chip, SparseCore, ICI, or network fabric faults. Use when multi-slice TPU training jobs freeze without progressing steps, emit `HANG_DETECTED`, or stall in collective operations. Don't use for gradual step-time throughput drops without hangs (use gke-ai-troubleshooting-tpu-performance-degradation) or pod preemption/eviction restarts (use gke-ai-troubleshooting-jobset-interruption).