Bump llguidance to 0.6.5

[hudson-ai]

Performance optimizations, multipleOf in JSON

[mmoskal]

We could do 0.6.2 - I added inline JSON in lark syntax, not really relevant for usage inside of guidance but doesn't hurt

[hudson-ai]

We could do 0.6.2 - I added inline JSON in lark syntax, not really relevant for usage inside of guidance but doesn't hurt

Sure! Currently getting a failure with gpt2 and phi2 in the tool call test though -- will take a look tomorrow:

FAILED tests/model_integration/library/test_gen.py::test_tool_call - ValueError: parser error: Too many items (limit 50000; mask); try avoiding single-byte/short lexemes

[mmoskal]

Well, the error message seems right

    @guidance(
        stateless=True, dedent=False
    )  # stateless=True indicates this function does not depend on LLM generation state
    def number(lm):
        n = one_or_more(select(["0", "1", "2", "3", "4", "5", "6", "7", "8", "9"]))
        # Allow for negative or positive numbers
        return lm + select(["-" + n, n])

Maybe we can slap regex() or however it's called around it?

See also guidance-ai/llguidance#9

[hudson-ai]

@mmoskal just out of curiosity, how do we hit 50_000 items so quickly? It seems to happen after only a few tokens have been generated, e.g. Calculator(1

[mmoskal]

Do the Tokenizers here have lots of number tokens? That would do it

[hudson-ai]

I can change the test to use a regex, but I am reluctant to do so given that this test is almost directly from our docs. We (1) should change the docs and (2) should automatically simplify grammars into lexemes when we can -- will leave this broken for a minute.