LLM Service

class app.services.llm_service.LLMService(chat_model=None, max_retries=3, timeout=30, **model_params)[source]

Bases: object

A service class for interacting with the Groq language model.

This class provides methods to generate responses using the Groq API, including standard responses, raw responses, and streaming responses. Raw responses and streaming responses were used for testing purposes, but are not used in the app.

Parameters:

  • chat_model – An optional ChatGroq instance to use instead of creating a new one.

  • max_retries – Maximum number of retries for API calls.

  • timeout – Timeout for API calls in seconds.

  • model_params – Additional parameters to pass to the ChatGroq constructor.

async generate_response(prompt: str, system_prompt: str = 'You are a helpful assistant specializing in window manufacturing.', **kwargs) → str[source]

Generate a response using the Groq chat model.

This method uses retry logic to handle potential failures.

Parameters:

  • prompt – The user’s input prompt.

  • system_prompt – The system prompt to set the context for the AI.

  • kwargs – Additional keyword arguments to pass to the chat model.

Returns:

The generated response as a string.

Raises:

ValueError – If the response type is unexpected.

async generate_response_raw(prompt: str, system_prompt: str = 'You are a helpful assistant specializing in window manufacturing.', **kwargs) → str[source]

Generate a raw response using the Groq chat model.

This method returns the raw text from the model’s response.

Parameters:

  • prompt – The user’s input prompt.

  • system_prompt – The system prompt to set the context for the AI.

  • kwargs – Additional keyword arguments to pass to the chat model.

Returns:

The raw generated response as a string.

generate_response_stream(prompt: str, system_prompt: str = 'You are a helpful assistant specializing in window manufacturing.', **kwargs) → AsyncGenerator[str, None][source]

Generate a streaming response using the Groq chat model.

This method returns an asynchronous generator that yields response chunks.

Parameters:

  • prompt – The user’s input prompt.

  • system_prompt – The system prompt to set the context for the AI.

  • kwargs – Additional keyword arguments to pass to the chat model.

Returns:

An asynchronous generator yielding response chunks.