Intel Core Ultra NPU runs Whisper locally

Whisper NPU is a compelling proof-of-concept that finally puts the "idle" NPU in modern laptops to productive use.

• –Offloads Whisper transcription to the NPU, preserving CPU and GPU cycles for intensive multitasking
• –Utilizes OpenVINO and INT4 quantization to achieve performance levels (8 to 26 TPS) suitable for real-time dictation
• –System-wide integration via global hotkey (Win+/) allows for seamless use across all Windows applications
• –Completely local execution ensures zero data leakage and works without an internet connection
• –Supports multiple model sizes, including large-v3-turbo, providing a scalable trade-off between speed and accuracy