Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: Set ToolCallBehavior.allowAllKernelFunctions(false) doesn't work for getStreamingChatMessageContentsAsync #279

Open
thomaspeng90s opened this issue Dec 9, 2024 · 1 comment
Labels
bug Something isn't working triage Needs triage from engineering team

Comments

@thomaspeng90s
Copy link

Describe the bug
I try to invoke toolcall myself in a streaming chat.
I have learned that how to invoke a toolcall in an non-streaming chat. like

        Kernel kernel = Kernel.builder().withAIService(ChatCompletionService.class, chatCompletionService)
                .withPlugin(KernelPluginFactory.createFromObject(this, "time"))
                .build();

        chatCompletionService
                .getChatMessageContentsAsync("what time is it now?", kernel, InvocationContext.builder()
                        .withToolCallBehavior(ToolCallBehavior.allowAllKernelFunctions(false))
                        .build())
                .block()
                .stream().filter(it->it instanceof OpenAIChatMessageContent<?>)
                .map(it->((OpenAIChatMessageContent)it))
                .forEach(it->{
                    System.out.println(it.getToolCall().size());
                });

Image

As you can see, I can get the toolCall value;

but when I change to an streaming chat, like

        Kernel kernel = Kernel.builder().withAIService(ChatCompletionService.class, chatCompletionService)
                .withPlugin(KernelPluginFactory.createFromObject(this, "time"))
                .build();

        chatCompletionService
                .getStreamingChatMessageContentsAsync("what time is it now?", kernel, InvocationContext.builder()
                        .withToolCallBehavior(ToolCallBehavior.allowAllKernelFunctions(false))
                        .build())
                .filter(it->it instanceof OpenAIStreamingChatMessageContent<?>)
                .map(it->(OpenAIStreamingChatMessageContent)it)
                .doOnNext(it->{
                    System.out.println(it.getToolCall().size());
                }).subscribe();

Image
the toolCall will always be an empty array.

@thomaspeng90s thomaspeng90s added bug Something isn't working triage Needs triage from engineering team labels Dec 9, 2024
@johnoliver
Copy link
Member

Support for tool calling in a streaming chat call is a feature that is still to be implemented, we have had some requests for support, will update if we put this on the immediate roadmap.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage Needs triage from engineering team
Projects
None yet
Development

No branches or pull requests

2 participants