-
Notifications
You must be signed in to change notification settings - Fork 35
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[libshortfin] Rollup of API changes and fixes to support the LLM serv…
…er. (#198) Core: * Convert Queue to shared ownership. * Allow creation of anonymous Queues. * Adds asyncio loop support for `call_at` * Fixes `VoidFuture::set_success` to also signal callbacks * Adds `StaticProgramParameters` and ability to construct a `ParameterProvider` * Fixes the hacky `Account::OnSync()` to return a `VoidFuture` and use it to route any wait error (instead of aborting) Bindings: * Adds `name` and `compute_dense_nd_size` bindings to `DType` * Makes `Process` subclassing more natural by supporting split new/init (allows arbitrary arguments to the subclass constructor) * Disables nanobind leak detector in debug builds of CPython (it seems to rely on immortalization of certain identifiers which are properly cleaned up in debug builds but left allocated forever in release builds) * Initialize logging from environment on load * Adds repr to `ProgramInvocation` * Reworks `Scope.devices` to support iteration and act like a built-in * Adds `Queue.write_nodelay` and `Queue.closed` * Adds `VoidFuture.set_success()` * Reworks optional dep handling to be more verbose/precise * Adds a module to perfom logging setup to the native logger FastAPI interop: * Replace `close_with_error` with `ensure_response` that can be used universally in a finally block. * Auto casts responses of `bytes` to a `Response`
- Loading branch information
1 parent
5a48398
commit 89d5d52
Showing
30 changed files
with
659 additions
and
180 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.