Skip to content

Commit

Permalink
link to impl docs
Browse files Browse the repository at this point in the history
  • Loading branch information
mmoskal committed Nov 9, 2024
1 parent 2366e7b commit 1180f44
Showing 1 changed file with 5 additions and 3 deletions.
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ The integration is ongoing in:
note that llama.cpp is fully integrated in Guidance above
via Python bindings

## Technical details

Given a context-free grammar, a tokenizer, and prefix of tokens,
llguidance computes a token mask (set of tokens from the tokenizer)
that when added to current prefix of token can lead to a valid string in
Expand All @@ -45,9 +47,9 @@ There is also no significant startup cost.

The library implements a context-free grammar parser with Earley's algorithm
on top of a lexer which uses [derivatives of regular expressions](https://github.com/microsoft/derivre).

Grammars are normally [JSON-serialized](./parser/src/api.rs).
The following libraries produce llguidance grammars:
A lot of
[low-level optimizations](https://github.com/microsoft/toktrie/blob/main/implementation.md)
are implemented.

## Building

Expand Down

0 comments on commit 1180f44

Please sign in to comment.