Last week at MVP Summit I had the pleasure of joining a roundtable with the Microsoft Foundry Local team. It was a genuinely productive session, the kind that leaves you scribbling notes on the when you get home, and it’s sparked a deeper interest in what Foundry Local can actually do for businesses in the real world. This is the first of what will be several posts as I dig in properly.

Every time your application sends a question to a cloud AI model, three things happen: your data leaves your building, you wait for a round trip over the internet, and you get charged for it.

For many use cases that’s a perfectly reasonable trade-off. But for businesses handling sensitive data, operating in regulated industries, or tired of unpredictable AI bills, it’s a deal breaker.

Foundry Local is Microsoft’s solution. It lets you run AI models directly on your own hardware, your servers, your laptops, your on-premises infrastructure; with none of the cloud dependency.

What Does “Running AI Locally” Actually Mean?

Think of traditional cloud AI like a taxi service. Every time your application needs to think (analyse a document, triage a ticket, transcribe a call) it hails a cab to a remote data centre, gets the answer, and comes back. You pay per journey.

Foundry Local is more like buying a car. You install the AI on your own hardware once, and from that point every query runs entirely on your machine. No internet required. No data leaving your site. No per-query bill.

The Business Case

Data never leaves your building. Every query, whether it’s a customer name, a contract clause, or a patient note, stays within your own infrastructure. No third-party processing, no data leaving your jurisdiction. For businesses in healthcare, legal, finance, or the public sector, this is often a compliance requirement, not just a preference.

No more unpredictable AI bills. Cloud AI is priced per word processed. For high volume workloads like processing every inbound email or triaging every support ticket, costs compound quickly and become hard to forecast. Foundry Local runs on hardware you already own. Once the model is downloaded, every additional query costs you nothing.

Speed that actually matters. Cloud AI introduces latency of a second or more per request. For real time use cases such as quality inspection on a production line, live call analysis, or instant document review, that delay creates real problems. Running locally, responses arrive in milliseconds.

Works without internet. For field teams, remote sites, factory floors, or anywhere network reliability can’t be guaranteed, Foundry Local works entirely offline once models are downloaded.

What Can You Run?

Local AI no longer means inferior AI. Foundry Local supports a catalogue of highly capable models — including Microsoft’s own Phi-4 — optimised for focused tasks like document analysis, classification, and summarisation. Voice-to-text transcription via Whisper is also supported, enabling speech-driven applications that process audio entirely on your own hardware.

For workloads that need more power, you can keep those in the cloud while keeping sensitive or high-volume tasks local. Both share the same APIs, so the choice of where something runs is a configuration decision, not a rebuild.

Real-World Scenarios

So how could this translate to some industries:

• Legal and professional services: contract review and document analysis without sending client data to a third-party service
• Healthcare: clinical note summarisation and transcription that keeps patient data inside the organisation, supporting GDPR compliance
• Manufacturing: real-time visual inspection on production lines where millisecond responses are required
• Financial services: transaction classification and report analysis where data residency rules prevent use of public cloud AI
• Customer support: high-volume ticket triage where per-token cloud pricing becomes unviable at scale

How Hard Is It to Adopt?

If your team can build web applications, they can run Foundry Local. It installs on Windows and macOS, integrates with Visual Studio Code, and exposes a standard API that any developer familiar with modern tooling will recognise. Foundry Local itself is free; the main investment is integration time, typically measured in days for well defined use cases.

The Bigger Picture

Foundry Local is part of Microsoft’s broader Foundry platform, which is the same ecosystem powering Azure AI at enterprise scale. It’s Microsoft’s deliberate answer to businesses that want the benefits of modern AI without ceding control of their data or their infrastructure.

The cloud remains the right answer for many workloads. But where data sovereignty, cost predictability, latency, or offline resilience matter, Foundry Local changes what’s possible and doesn’t make you to choose between capability and control.

This is just the start. Over the coming weeks/months I’ll be going hands on with Foundry Local. Exploring the model catalogue, testing real world integration patterns, and looking at how it fits into broader AI architectures. Stay tuned for more.