Added AI Agent Semantic Convention #1739
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| # Use this changelog template to create an entry for release notes. | ||
| # | ||
| # If your change doesn't affect end users you should instead start | ||
| # your pull request title with [chore] or use the "Skip Changelog" label. | ||
|
|
||
| # One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix' | ||
| change_type: enhancement | ||
|
|
||
| # The name of the area of concern in the attributes-registry, (e.g. http, cloud, db) | ||
| component: gen-ai | ||
|
|
||
| # A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`). | ||
| note: Added AI Agent Semantic Convention | ||
|
|
||
| # Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists. | ||
| # The values here must be integers. | ||
| issues: [1732] | ||
|
|
||
| # (Optional) One or more lines of additional information to render under the primary note. | ||
| # These lines will be padded with 2 spaces and then inserted directly into the document. | ||
| # Use pipe (|) for multiline entries. | ||
| subtext: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -17,6 +17,9 @@ This document defines the attributes used to describe telemetry in the context o | |
|
|
||
| | Attribute | Type | Description | Examples | Stability | | ||
| |---|---|---|---|---| | ||
| | <a id="gen-ai-agent-description" href="#gen-ai-agent-description">`gen_ai.agent.description`</a> | string | Free-form description of the GenAI agent provided by the application. | `Helps with math problems`; `Generates fiction stories` |  | | ||
| | <a id="gen-ai-agent-id" href="#gen-ai-agent-id">`gen_ai.agent.id`</a> | string | The unique identifier of the GenAI agent. | `asst_5j66UpCpwteGg4YSxUnt7lPY` |  | | ||
| | <a id="gen-ai-agent-name" href="#gen-ai-agent-name">`gen_ai.agent.name`</a> | string | Human-readable name of the GenAI agent provided by the application. | `Math Tutor`; `Fiction Writer` |  | | ||
| | <a id="gen-ai-operation-name" href="#gen-ai-operation-name">`gen_ai.operation.name`</a> | string | The name of the operation being performed. [1] | `chat`; `text_completion`; `embeddings` |  | | ||
| | <a id="gen-ai-request-encoding-formats" href="#gen-ai-request-encoding-formats">`gen_ai.request.encoding_formats`</a> | string[] | The encoding formats requested in an embeddings operation, if specified. [2] | `["base64"]`; `["float", "binary"]` |  | | ||
| | <a id="gen-ai-request-frequency-penalty" href="#gen-ai-request-frequency-penalty">`gen_ai.request.frequency_penalty`</a> | double | The frequency penalty setting for the GenAI request. | `0.1` |  | | ||
|
|
@@ -33,6 +36,9 @@ This document defines the attributes used to describe telemetry in the context o | |
| | <a id="gen-ai-response-model" href="#gen-ai-response-model">`gen_ai.response.model`</a> | string | The name of the model that generated the response. | `gpt-4-0613` |  | | ||
| | <a id="gen-ai-system" href="#gen-ai-system">`gen_ai.system`</a> | string | The Generative AI product as identified by the client or server instrumentation. [3] | `openai` |  | | ||
| | <a id="gen-ai-token-type" href="#gen-ai-token-type">`gen_ai.token.type`</a> | string | The type of token being counted. | `input`; `output` |  | | ||
| | <a id="gen-ai-tool-call-id" href="#gen-ai-tool-call-id">`gen_ai.tool.call.id`</a> | string | The tool call identifier. | `call_mszuSIzqtI65i1wAUOE8w5H4` |  | | ||
|
There was a problem hiding this comment. Could we consider recording the tools available to the agents or simple the models? This is also requested in this issue: #1760 There was a problem hiding this comment. I think this should belong to There was a problem hiding this comment. The tool message event is different as it records the responses from invoking the tools. We currently don't have a way to record what tools are available to the model or agent to invoke. |
||
| | <a id="gen-ai-tool-name" href="#gen-ai-tool-name">`gen_ai.tool.name`</a> | string | Name of the tool utilized by the agent. | `Flights` |  | | ||
|
There was a problem hiding this comment. Models can sometimes call multiple tools in parallel. Maybe this needs to be a list? There was a problem hiding this comment. yes, let me check how to update this to a list. |
||
| | <a id="gen-ai-tool-type" href="#gen-ai-tool-type">`gen_ai.tool.type`</a> | string | Type of the tool utilized by the agent [4] | `function`; `extension`; `datastore` |  | | ||
| | <a id="gen-ai-usage-input-tokens" href="#gen-ai-usage-input-tokens">`gen_ai.usage.input_tokens`</a> | int | The number of tokens used in the GenAI input (prompt). | `100` |  | | ||
| | <a id="gen-ai-usage-output-tokens" href="#gen-ai-usage-output-tokens">`gen_ai.usage.output_tokens`</a> | int | The number of tokens used in the GenAI response (completion). | `180` |  | | ||
|
|
||
|
|
@@ -52,14 +58,18 @@ attribute may help identify the actual system in use for `openai`. | |
| For custom model, a custom friendly name SHOULD be used. | ||
| If none of these options apply, the `gen_ai.system` SHOULD be set to `_OTHER`. | ||
|
|
||
| **[4] `gen_ai.tool.type`:** Extension: A tool executed on the agent-side to directly call external APIs, bridging the gap between the agent and real-world systems. Function: A tool executed on the client-side, where the agent generates parameters for a predefined function, and the client executes the logic. Datastore: A tool used by the agent to access and query structured or unstructured external data for retrieval-augmented tasks or knowledge updates. | ||
|
There was a problem hiding this comment. I'm not clear on the what you mean by "agent-side" and "client-side". The term "agent-side" isn't used elsewhere in this document or the repo at large. I think they need defining better. More generally, I don't think your distinction between a function and a tool is how most people would understand them. That said, I don't have a good definition for the distinction. There was a problem hiding this comment. My suggestion would be:
Most tools are functions, but not all. There was a problem hiding this comment. okay, so I see now that "agent-side" is used in the google whitepaper, but again without a definition 😞 . |
||
|
|
||
| --- | ||
|
|
||
| `gen_ai.operation.name` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used. | ||
|
|
||
| | Value | Description | Stability | | ||
| |---|---|---| | ||
| | `chat` | Chat completion operation such as [OpenAI Chat API](https://platform.openai.com/docs/api-reference/chat) |  | | ||
| | `create_agent` | Create GenAI agent |  | | ||
|
There was a problem hiding this comment. do we need to trace create_agent? not that I have a strong opinion about this. But, it may not be that valuable. There was a problem hiding this comment. @karthikscale3 this is important if we have multiple agents, with multiple agents, the agents may have some strict sequence to create, to run etc. There was a problem hiding this comment. I agree that I haven't got a strong opinion on add this since it can simply be unused. |
||
| | `embeddings` | Embeddings operation such as [OpenAI Create embeddings API](https://platform.openai.com/docs/api-reference/embeddings/create) |  | | ||
| | `execute_tool` | Execute a tool |  | | ||
| | `text_completion` | Text completions operation such as [OpenAI Completions API (Legacy)](https://platform.openai.com/docs/api-reference/completions) |  | | ||
|
|
||
| --- | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we also want the instruction given to the agent?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, usually the instruction will act as prompt for the models.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@TaoChenOSU do you mean we need to add a new attribute for instruction?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, maybe in the create agent span. Some services embed the instruction into a large prompt, such as Bedrock agent.