Building a Local AI Agent with .NET and Ollama — Getting Started

AI is everywhere these days, but most of the examples we see involve sending our data to a cloud-based AI service.

I wanted to try something a little different:

Can we build an AI agent that runs locally and can be integrated with a .NET application?

The answer is yes.

In this series, I’ll walk through how I’m building a local AI agent using .NET, C# and Ollama, and gradually give the agent more capabilities.

The goal isn’t just to create another chatbot. We will eventually make the agent capable of working with files, databases, APIs and other tools.

---

What Are We Building?

The basic idea is fairly simple. We will have a .NET application that communicates with a locally running AI model.

User Request

↓

AI Agent

↓

Agent Decision & Execution

1

Decide what needs to be done

2

Call a tool

3

Read the result

4

Think about the result

5

Continue if necessary

↓

Final Answer

Everything runs on our own machine.

That means we don’t need to send every prompt to an external AI API just to experiment with the technology.

Of course, whether a particular local model is powerful enough depends on your computer’s hardware and the model you choose.

---

Why Use a Local AI Model?

1. Privacy

If you’re experimenting with internal documents or development data, keeping the AI processing locally can be useful.

2. No API Key

For local experimentation, you don’t need to create an account with an AI provider just to start testing.

3. Learning

This is probably the biggest reason.

When everything runs locally, you get a much better understanding of what is actually happening between your application and the AI model.

4. Development Freedom

You can experiment with prompts, models, tools and agent workflows without worrying about API usage costs during every test.

---

What Is Ollama?

For this project, we’ll use Ollama to run the AI model locally.

Ollama makes it relatively easy to download and run different language models on your own computer.

Once Ollama is installed, you can download a model and run it from the command line.

ollama pull llama3.2

Then:

ollama run llama3.2

Now you have a local AI model running on your machine.

The exact model you choose can change as new models become available, so don’t consider the model name above a permanent requirement for the project.

---

Creating the .NET Project

I’m going to use ASP.NET Core for the application.

Create a new Web API project:

dotnet new webapi -n LocalAIAgent

Move into the project:

cd LocalAIAgent

And run it:

dotnet run

At this point, we have a normal ASP.NET Core application.

Nothing particularly exciting yet.

The interesting part comes when we connect it to our local AI model.

---

Calling Ollama from C#

Ollama exposes an HTTP API, which means we can communicate with it from our .NET application using the standard HttpClient.

For example:

using System.Net.Http.Json; public class OllamaService { private readonly HttpClient _httpClient; public OllamaService(HttpClient httpClient) { _httpClient = httpClient; } public async Task<string?> AskAsync(string prompt) { var request = new { model = "llama3.2", prompt = prompt, stream = false }; var response = await _httpClient.PostAsJsonAsync( "http://localhost:11434/api/generate", request); response.EnsureSuccessStatusCode(); var result = await response.Content .ReadFromJsonAsync<OllamaResponse>(); return result?.Response; } } public class OllamaResponse { public string? Response { get; set; } }

The important thing here is that we’re not calling a cloud AI service.

We’re simply making an HTTP request to a service running on our own computer.

---

Registering the Service

In Program.cs:

builder.Services.AddHttpClient<OllamaService>();

Now ASP.NET Core can inject our service wherever we need it.

We can then create a simple controller:

[ApiController] [Route("api/[controller]")] public class AIController : ControllerBase { private readonly OllamaService _ollama; public AIController(OllamaService ollama) { _ollama = ollama; } [HttpPost("ask")] public async Task<IActionResult> Ask(string prompt) { var response = await _ollama.AskAsync(prompt); return Ok(new { response }); } }

Now our .NET application can send a question to the local model and return the response.

---

But This Isn’t Really an AI Agent Yet

And this is an important distinction.

At this stage, we have something closer to a local AI chatbot.

An agent is more interesting.

Instead of simply doing this:

 Question → AI → Answer 

we want to eventually build something like:

User Request

↓

AI Agent

↓

Agent Decision & Execution

1

Decide what needs to be done

2

Call a tool

3

Read the result

4

Think about the result

5

Continue if necessary

↓

Final Answer

For example, imagine asking:

“Find the sales report for August and tell me which product had the highest sales.”

A normal chatbot may simply tell you that it doesn’t have access to your files.

Our agent could eventually:

1. Search a folder.
2. Find the relevant report.
3. Read the file.
4. Analyze the data.
5. Determine the highest-selling product.
6. Return the result.

That’s where things start getting interesting.

---

Where We Are Going Next

In the next parts of this series, I’ll extend this basic application step by step.

• Conversation memory
• System prompts
• Agent tools
• Reading local files
• Working with PDFs
• Connecting to SQL Server
• Calling external APIs
• RAG (Retrieval-Augmented Generation)
• Angular frontend
• Multi-step agent workflows
• Authentication and security

The final goal is to have a useful local AI assistant built around .NET, rather than just another chat window.

---

Why I’m Building This

AI development doesn’t have to mean abandoning the technologies we already use.

If you’re a .NET developer, you can bring AI into the applications you’re already comfortable building with C#, ASP.NET Core, SQL Server and Angular.

That’s what I want to explore through this series.

I’ll keep the examples practical and build the project incrementally so that we can see not only what works, but also why it works.

In the next article, we’ll connect our .NET application to Ollama properly and build our first working local AI chat endpoint.

Stay tuned.

---

Dinesh Wadhwa
IT Solutions
Full Stack Developer