refactor: ollama client api

[TobiasGleiter]

Refactor Ollama Client API

Currently, the Ollama Client API (llms/ollama/*.go) is poorly designed. We need to come up with a more production ready approach for handling API calls to the Ollama REST endpoints.

[sashithaf16]

Hi @TobiasGleiter

I took a look at the current implementation in llms/ollama and would love to help refactor this.

Before starting, I wanted to align on expectations and also share a few observations about the current design:

Current pain points (from what I can see)

• HTTP client is instantiated per request instead of being reused
• context.Context is not consistently used (e.g., Generate ignores it)
• Request construction and HTTP execution are tightly coupled
• Error handling is inconsistent and not wrapped (errors.New vs fmt.Errorf)
• Some duplication across methods (request creation, endpoint resolution, etc.)
• No unit tests

Proposed direction

I’m thinking of restructuring the client along these lines:

• Introduce a proper Client struct:

  • Holds a reusable http.Client
  • Configurable (timeouts, base URL, headers)

• Separate concerns:

  • Request builders (chat, embeddings, etc.)
  • Transport layer (HTTP execution)
  • Response parsing

• Improve context support:

  • Ensure all public methods accept and propagate context.Context

• Standardize error handling:

  • Use wrapped errors (fmt.Errorf("...: %w", err))

• Reduce duplication:

  • Shared internal request execution method

• Improve testability:

  • Introduce interfaces for easier mocking

If this direction makes sense, I can put together a draft PR to iterate on.

Happy to take this on 👍

[TobiasGleiter]

@sashithaf16, Thanks! Yes, this is definitely better than what it implemented right now.

I'm not sure about the interface you are speaking of. It might not ne necessary. Remember: Take interface, return struct.

Feel free to go ahead with the implementation!