bug: definition of json schema in prompt

[kdziedzic68]

What happened?

When we pass a pydantic.BaseModel as output type to the prompt without passing the schema itself to the system prompt, we are getting parsing errors - the desired structure is not preserved in raw LLM output even though using models that support that feature described here: https://platform.openai.com/docs/guides/structured-outputs .

How can we reproduce it?

import asyncio
from pydantic import BaseModel

from ragbits.core.prompt import Prompt
from ragbits.core.llms.litellm import LiteLLM


class QueryWithContext(BaseModel):
    """
    Input format for the QueryWithContext.
    """

    query: str
    context: list[str]


class OutputSchema(BaseModel):
    last: str
    previous: str




class RAGPrompt(Prompt[QueryWithContext, OutputSchema]):
    """
    A simple prompt for RAG system.
    """

    system_prompt = f"""
    You are a helpful assistant. Answer the QUESTION that will be provided using CONTEXT.
    If in the given CONTEXT there is not enough information refuse to answer.
    """

    user_prompt = """
    QUESTION:
    {{ query }}

    CONTEXT:
    {% for item in context %}
        {{ item }}
    {% endfor %}
    """


async def main():
    llm = LiteLLM(model_name="gpt-4o-2024-08-06")
    query = "Write down names of last two world cup winners"
    context = ["Today is November 2017", "Germany won 2014 world cup", "Spain won 2010 world cup"]
    prompt = RAGPrompt(QueryWithContext(query=query, context=context))
    response = await llm.generate(prompt)
    print(response)


asyncio.run(main())

Relevant log output

No response