How to avoid multiple consecutive calls to the same function in ToolCallBehavior.AutoInvokeKernelFunctions?

[LuisM000]

I'm using Semantic Kernel's ToolCallBehavior.AutoInvokeKernelFunctions to orchestrate a set of functions in my application. One of these functions is designed to perform operations on documents, such as changing their formats or processing their content.

Occasionally, the orchestrator plans multiple consecutive calls to this function, each targeting a single document, instead of consolidating them into a single call for all documents. For example, if the user requests an operation on two documents, the orchestrator sometimes schedules two separate calls to the function, even though it is capable of handling both in one.

I've experimented with modifying the prompts for both the orchestrator and the function's descriptions (including parameter definitions), but I haven't been able to consistently achieve the desired behavior where the requests are combined into a single invocation.

Question:
Is there a clean and non-hacky way to control this behavior and ensure that, when multiple consecutive calls to the same function are planned, they are consolidated into one? One idea I considered was using IAutoFunctionInvocationFilter to intercept and manage the calls, but I'm not sure if that's the most appropriate solution.

Any advice or recommendations would be greatly appreciated!

[moonbox3]

tagging @SergeyMenshykh for visibility

[SergeyMenshykh]

Hi @LuisM000, which model do you use?

[LuisM000]

Hi @SergeyMenshykh!
I am using Azure OpenAI gpt-4o (2024-08-06).

[SergeyMenshykh]

@LuisM000, please consider the following:

1. Explicitly enable parallel calls by setting the FunctionChoiceBehaviorOptions.AllowParallelCalls property to true, as described here - sample1 or here sample2. Parallel function calling is enabled by default for {Azure} OpenAI models, as far as we know, and setting it explicitly might make your scenario more consistent if, for example, the inconsistency is caused by some faulty Azure nodes/VMs handling a certain percentage of your requests where, for whatever reason, parallel function calling is disabled by default. You will have to update your code to use the new function calling API represented by FunctionChoiceBehavior classes, as described here - https://learn.microsoft.com/en-us/semantic-kernel/support/migration/function-calling-migration-guide?pivots=programming-language-csharp, to be able to set that option explicitly.

2. If your scenario allows, try to change the function signatures to indicate to the LLM that they can work with multiple documents instead of one. For example, if you have the ChangeDocumentFormat(string documentId, ....) function, change it to ChangeDocumentsFormat(string[] documentIds, ....). The idea here is that the LLM makes only one call for all documents.

3. Revise your scenario to ensure that there is no dependency between the documents. For example, ensure that the input for the function to handle the second document does not require output from the first function. Most likely, this is not the case, but it is worth double-checking.

4. Just an idea to try to get to the root cause of the behavior: ask the LLM to explain the reason the function was not called in parallel. This can be done by temporarily adding an extra parameter of string type - ChangeDocumentFormat(string documentId, ..., [Description("Reason the function was not called in parallel")] string reason) to the function that is occasionally called sequentially. Check the reason and decide on the next steps based on this. We have never tried it, but we are interested in the results of this experiment.

[LuisM000]

Hi @SergeyMenshykh,
Thank you for the detailed response!

1. I’ll review the first point, though I’m not entirely sure it matches my exact scenario. My issue is primarily that calls aren’t being consolidated into one. Nonetheless, I’ll look into it :)

2. I’ve been quite explicit with the parameters and have refined the descriptions for both the functions and the orchestrator. This has reduced erratic behavior, and in most cases, calls are consolidated, but not always.

3. Good point! I’ve checked, and everything seems fine on that front.

4. Great idea! Thanks! I’ll give it a try to see if it provides insights into why the calls are being split.

My question was more about whether there’s a programmatic (i.e., deterministic) way to address this behavior.

Thanks again for your help!

[SergeyMenshykh]

@LuisM000, we haven't heard of any programmatic way to influence the LLM's behavior.

Keep us updated, and feel free to share your function signatures/descriptions so we can take a look at them.

[LuisM000]

Hi @SergeyMenshykh,
Thanks for your reply.
It seems there’s been some misunderstanding. My thoughts were more along the lines of controlling this behavior within Semantic Kernel itself. Since I’m using AutoInvokeKernelFunctions, and similar to how you can intervene in the flow of execution (e.g., using IAutoFunctionInvocationFilter to terminate calls with context.Terminate), I was wondering if it’s possible to intervene and consolidate multiple sequential calls to the same function into a single call, in cases where such sequential executions are planned.
But I understand now that this might not be possible for now.

Thanks again, and I’ll continue sharing any findings or tests I make along the way! :)

[SergeyMenshykh]

@LuisM000, got you. You can access function calls if you enable manual function invocation. Then you will need to consolidate the functions arguments and invoke the function yourself. Additionally, you will need to remove the original function calls and add a simulated function call and return your function invocation result as part of that simulated call.

That will only work if the function calls came as part of one request/response from LLM.

[LuisM000]

Great, thanks a lot! I now have all the information I need :)

[SergeyMenshykh]

@LuisM000, you are welcome. I am closing the discussion for now. Feel free to reopen this or start a new one if needed.

[SergeyMenshykh]

@LuisM000