⚠️ This blog post was created with the help of AI tools. Yes, I used a bit of magic from language models to organize my thoughts and automate the boring parts, but the geeky fun and the 🤖 in C# are 100% mine.

Hi!

Avoid reading the blog post with this 5-min video:

When working with local models in OllamaSharp, I hit a timeout while running long-running workloads like video analysis. The issue wasn’t the model, it was the default 100-second timeout coming from HttpClient.


The problem

By default, OllamaSharp uses an HttpClient with a fixed timeout.
If your model needs more time, you’ll see errors like:

Unhandled exception. System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
 ---> System.TimeoutException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
 ---> System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.InitialFillAsync(Boolean async)

What I learned

The timeout must be configured on the HttpClient, before passing it to the OllamaSharp client. That works—but it’s not very clean or discoverable.


The solution: a C# extension

I created a small extension library so you can do this instead:

ollamaClient.SetTimeout(TimeSpan.FromMinutes(5));

There are also convenience helpers:

// Sets a timeout suitable for quick queries (2 minutes).
ollamaClient.WithQuickTimeout(); 
// Sets a timeout suitable for standard prompts (5 minutes).
ollamaClient.WithStandardTimeout();
// Sets a timeout suitable for long-form generation (10 minutes).
ollamaClient.WithLongTimeout();


Get it here


Takeaway

If something hurts in C#…
it probably deserves an extension method 😄

Happy coding!

Greetings

El Bruno

More posts in my blog ElBruno.com.

More info in https://beacons.ai/elbruno


Leave a comment

Discover more from El Bruno

Subscribe now to keep reading and get access to the full archive.

Continue reading