Foundry Local

Foundry Local is a set of native SDKs that let you download, cache, load, and call optimized AI models entirely on a user's device. Instead of sending data to a cloud API, you keep inference in-process — on CPU, GPU, or NPU — and the SDK handles hardware selection for you. It supports Python, JavaScript, C#, and Rust, so you can prototype quickly and ship production apps without rewriting your stack.

The SDK lifecycle is simple: create a manager, pick a model from a catalog, download it once, load it, and then call chat or audio clients directly from your app. There's no need to manage execution providers or fallbacks — the SDK picks the best hardware available (NPU, GPU, or CPU) automatically. Models and runtime are cached locally, so features work offline. And if you ever need a REST endpoint, you can start an optional OpenAI-compatible server to plug into frameworks like LangChain.

This tool is ideal for developers building privacy-sensitive apps — healthcare, finance, personal assistants — where data must never leave the device. It also works well for mobile and desktop apps that need fast, reliable AI without depending on network latency. If you want to embed LLM chat, audio processing, or other on-device AI features without managing infrastructure, Foundry Local gives you a clean, SDK-first approach.