跳到主要内容

api_debug

· 阅读需 3 分钟

Input:

[
{
"type": "message",
"role": "user",
"content": "What tools do you have now?"
},
{
"id": "rs_0fdd74348bc35ef30068f737e4176c8193a80e94c4ec936852",
"summary": [],
"type": "reasoning",
"content": null,
"encrypted_content": null
},
{
"id": "msg_0fdd74348bc35ef30068f737e448448193839077101c450ab6",
"content": [
{
"annotations": [],
"text": "I have access to a tool called \u201cget_weather,\u201d which lets me look up the current weather for a specified city. If you give me a city name, I can retrieve its current weather conditions for you.",
"type": "output_text",
"logprobs": []
}
],
"role": "assistant",
"type": "message"
},
{
"type": "message",
"role": "user",
"content": "What's the time in Shanghai right now? call the tool please"
}
]

Response:

Item with id 'rs_0fdd74348bc35ef30068f737e4176c8193a80e94c4ec936852' not found.

Input

[
{
"type": "message",
"role": "user",
"content": "What tools do you have now?"
},
{
"id": "msg_000d06e3ba7d8b460068f739af967c819097e8d97431e34655",
"content": [
{
"annotations": [],
"text": "Here\u2019s what I can use:\n\n1. Image input \u2013 I can receive and reason about images you provide.\n2. A \u201cget_weather\u201d tool \u2013 allows me to look up the current weather for a city you specify.",
"type": "output_text",
"logprobs": []
}
],
"role": "assistant",
"type": "message"
},
{
"type": "message",
"role": "user",
"content": "What's the time in Shanghai right now? call the tool please"
}
]

Response:

Item 'msg_000d06e3ba7d8b460068f739af967c819097e8d97431e34655' of type 'message' was provided without its required 'reasoning' item: 'rs_000d06e3ba7d8b460068f739aefd8c8190959e03354167b214'.

解决方案:丢弃上一轮的 reasoning item;且删除或替换 reasoning item 后的 FunctionCallItem 或 Assistant Message Item 的 id(只用删除 id 或者把 id 改成另外一个 id,只要不是之前的 id 就行,不需要删除整个 item)。

首先,这类错误只会在多轮场景下触发,原因是 openai 在接口内就对上一轮的 reasoning item 做了丢弃 (https://platform.openai.com/docs/guides/conversation-state?api-mode=responses),导致上一轮的 reasoning item id 找不到。如果手动丢弃上一轮的 reasoning item,但 openai 接口侧还是会缓存 assistant message item 的 id 和 reasoning item 的 id 的匹配,也会报错。继续把上一轮的 assistant message item 的 id 也删除掉,就不会再报错了。另一个场景是 reasoning 然后 function call 时也会触发类似的错误,解决方案也是删除上一轮的 reasoning item 和 function call item 的 id。当然,function call item 和 assistant message item 的 id 还是有作用的,更好的做法是替换为另外一个 id,而不是直接删除掉 id。

关于 openai 接口侧会缓存 assistant message item 的 id 和 reasoning item 的 id 的匹配,是因为推理引擎里需要做 prefix cache 和 kv cache,而 openai 的 reasoning item 的原始内容是不给你的,只给你 summary 的结果,它需要通过 reasoning item id 来找到对应的原始 kv cache 以便命中缓存。

如何设计一个有状态的流式响应系统

· 阅读需 18 分钟
Xueyuan Lin
Agentlin maintainer

在构建对话式AI系统时,设计一个能够处理有状态流式响应的系统是至关重要的。本文将探讨如何设计一个有状态的流式响应系统,以提升用户体验和系统性能。

流式传输

首先确定系统的最终状态是输出一个结构化的大 json 对象。 其次,系统应该以流式方式逐步组装出这个大对象,在每一步都能向用户展示当前的进展和进度状态。

终点状态

系统的最终状态是一个完整的 json 对象。我们考虑 4 种消息类型:reasoning,tool_call,tool_result,message。

{
"task_id": "task_1234xyz",
"output": [
{
"type": "reasoning",
"id": "rs_1234xyz",
"summary": [
{
"type": "text",
"text": "Thinking about the weather in Paris."
},
{
"type": "text",
"text": "Decided to call get_weather function."
}
],
},
{
"type": "tool_call",
"id": "fc_1234xyz",
"call_id": "call_1234xyz",
"name": "get_weather",
"arguments": "{\"location\":\"Paris, France\"}",
},
{
"type": "tool_result",
"id": "fco_1234xyz",
"call_id": "call_1234xyz",
"content": [
{
"type": "text",
"text": "<referencable-item>\nID: 1",
"id": 1,
"tags": ["added_by_reference_manager"],
}, // <- 添加引用池标记
{
"type": "text",
"text": "{\"temperature\":\"15C\",\"condition\":\"Sunny\"}",
"id": 1,
},
{
"type": "image_url",
"image_url": {
"url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
},
"id": 1,
},
{
"type": "text",
"text": "</referencable-item>",
"id": 1,
"tags": ["added_by_reference_manager"],
} // <- 添加引用池标记
],
"block_list": [
{
"type": "text",
"text": "{\"temperature\":\"15C\",\"condition\":\"Sunny\"}",
"id": 1,
},
{
"type": "image_url",
"image_url": {
"url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
},
"id": 1,
}
]
},
{
"type": "message",
"id": "msg_1234xyz",
"role": "assistant",
"content": [
{
"type": "text",
"text": "The weather in Paris is sunny with a temperature of 15C.[^1]",
}
],
"block_list": [
{
"type": "text",
"text": "The weather in Paris is sunny with a temperature of 15C.[^1]",
"id": 1,
"annotations": [
{
"type": "reference_to_block",
"reference_id": 1,
"start_index": 44,
"end_index": 47
},
]
},
]
},
]
}

子 agent 作为工具的场景下,子 agent 的输出也对应一个 task_id,但会经过处理:

  1. 只有 message 部分加上引用池会嵌入到主 agent 的 tool_result 的 content 里
  2. 所有部分会转换为 block 嵌入到主 agent 的 tool_result 的 block_list 里。 所以主 agent 只看到子 agent的结果 message,而用户能看到子 agent 的包括思考和工具调用的调用过程。

event 设计

首先,所有的 event 都是围绕 task_id 进行的;所有的 event 都有严格的类型定义。所以,所有的 event 都有两个核心字段:typetask_id

根据 event 的类型不同,设计不同的 event 结构。

我们主要面向 agent 的输出,而 agent 的输出包含 4 中类型的 output item:reasoning,tool_call,tool_result,message。 因此,我们设计 4 种 output item 的流式传输方式。

ReasoningItem 流式传输设计

考虑以下 reasoning item:

{
"type": "reasoning",
"id": "rs_1234xyz",
"summary": [
{
"type": "text",
"text": "Thinking about the weather in Paris."
},
{
"type": "text",
"text": "Decided to call get_weather function."
}
],
}

对于 summary 字段,由于是一个列表,我们可以将列表拆分成多个 item 进行流式传输。每个 item 内部如果有长文本,也可以继续拆分成 delta 进行流式传输。

// Added
{"type":"task.output_item.added","task_id":"task_1234xyz","output_index":0,"item":{"type":"reasoning","id":"rs_1234xyz","summary":[]}}
// First summary item
{"type":"task.reasoning_summary_item.added","task_id":"task_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"item":{"type":"text","text":""}}
{"type":"task.reasoning_summary_text.delta","task_id":"task_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"delta":"Thinking about the weather "}
{"type":"task.reasoning_summary_text.delta","task_id":"task_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"delta":"in Paris."}
{"type":"task.reasoning_summary_item.done","task_id":"task_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"item":{"type":"text","text":"Thinking about the weather in Paris."}}
// Second summary item
{"type":"task.reasoning_summary_item.added","task_id":"task_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"item":{"type":"text","text":""}}
{"type":"task.reasoning_summary_text.delta","task_id":"task_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"delta":"Decided to call "}
{"type":"task.reasoning_summary_text.delta","task_id":"task_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"delta":"get_weather function."}
{"type":"task.reasoning_summary_item.done","task_id":"task_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"item":{"type":"text","text":"Decided to call get_weather function."}}
// Done
{"type":"task.output_item.done","task_id":"task_1234xyz","output_index":0,"item":{"type":"reasoning","id":"rs_1234xyz","summary":[
{"type":"text","text":"Thinking about the weather in Paris."},
{"type":"text","text":"Decided to call get_weather function."}
]}}

ToolCallItem 流式传输设计

考虑以下 tool_call item:

{
"type": "tool_call",
"id": "fc_1234xyz",
"call_id": "call_1234xyz",
"name": "get_weather",
"arguments": "{\"location\":\"Paris, France\"}",
}

对于短的文本(如 name),可以直接传,不需要拆分成 delta。对于长文本(如 arguments),需要拆分成 delta 进行流式传输。

// Added
{"type":"task.output_item.added","task_id":"task_1234xyz","output_index":1,"item":{"type":"tool_call","id":"fc_1234xyz","call_id":"call_1234xyz","name":"get_weather","arguments":""}}
// Deltas for arguments
{"type":"task.tool_call_arguments.delta","task_id":"task_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"{\""}
{"type":"task.tool_call_arguments.delta","task_id":"task_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"location"}
{"type":"task.tool_call_arguments.delta","task_id":"task_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"\":\""}
{"type":"task.tool_call_arguments.delta","task_id":"task_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"Paris"}
{"type":"task.tool_call_arguments.delta","task_id":"task_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":","}
{"type":"task.tool_call_arguments.delta","task_id":"task_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":" France"}
{"type":"task.tool_call_arguments.delta","task_id":"task_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"\"}"}
{"type":"task.tool_call_arguments.done","task_id":"task_1234xyz","item_id":"fc_1234xyz","output_index":1,"arguments":"{\"location\":\"Paris, France\"}"}
// Done
{"type":"task.output_item.done","task_id":"task_1234xyz","output_index":1,"item":{"type":"tool_call","id":"fc_1234xyz","call_id":"call_1234xyz","name":"get_weather","arguments":"{\"location\":\"Paris, France\"}"}}

ToolResultItem 流式传输设计

我们假设工具的输出为一个包含 message_content 和 block_list 的结构化对象。

{
"message_content": [
{
"type": "text",
"text": "{\"temperature\":\"15C\",\"condition\":\"Sunny\"}",
"id": 1,
},
{
"type": "image_url",
"image_url": {
"url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
},
"id": 1,
},
],
"block_list": [
{
"type": "text",
"text": "{\"temperature\":\"15C\",\"condition\":\"Sunny\"}",
"id": 1,
},
{
"type": "image_url",
"image_url": {
"url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
},
"id": 1,
}
]
}

根据引用协议,将会被处理为以下 tool_result item:

{
"type": "tool_result",
"id": "fco_1234xyz",
"call_id": "call_1234xyz",
"content": [
{
"type": "text",
"text": "<referencable-item>\nID: 1",
"id": 1,
"tags": ["added_by_reference_manager"],
}, // <- 添加引用池标记
{
"type": "text",
"text": "{\"temperature\":\"15C\",\"condition\":\"Sunny\"}",
"id": 1,
},
{
"type": "image_url",
"image_url": {
"url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
},
"id": 1,
},
{
"type": "text",
"text": "</referencable-item>",
"id": 1,
"tags": ["added_by_reference_manager"],
} // <- 添加引用池标记
],
"block_list": [
{
"type": "text",
"text": "{\"temperature\":\"15C\",\"condition\":\"Sunny\"}",
"id": 1,
},
{
"type": "image_url",
"image_url": {
"url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
},
"id": 1,
}
]
}

给前端只传 block_list,所以只需要将 block_list 处理为以下流式传输事件:

// Added: 初始状态下 content 和 block_list 都是空的
{"type":"task.output_item.added","task_id":"task_1234xyz","output_index":2,"item":{"type":"tool_result","id":"fco_1234xyz","call_id":"call_1234xyz","block_list":[]}}
// Deltas for block_list
// 1. 对于较短的 block,一次性传输
{"type":"task.text.done","task_id":"task_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":0,"item":{"type":"text","text":"{\"temperature\":\"15C\",\"condition\":\"Sunny\"}","id":1}}
// 2. 对于较大的 block,带上状态进行传输
{"type":"task.image.added","task_id":"task_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":""},"id":1}}
{"type":"task.image.done","task_id":"task_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
// 3. 对于超大的 block,可以继续拆分 delta 进行传输
// image - added 前端开始准备传输
{"type":"task.image.added","task_id":"task_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":""},"id":1}}
// image - deltas 传输中,每个 delta 都是一个完整的图片
{"type":"task.image.delta","task_id":"task_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"partial_image_index":0,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
{"type":"task.image.delta","task_id":"task_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"partial_image_index":1,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
// image - done 传输完成,最后一个图片才是最终状态的图片
{"type":"task.image.done","task_id":"task_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
// Done
{"type":"task.output_item.done","task_id":"task_1234xyz","output_index":2,"item":{"type":"tool_result","id":"fco_1234xyz","call_id":"call_1234xyz","block_list":[
{"type":"text","text":"{\"temperature\":\"15C\",\"condition\":\"Sunny\"}","id":1},
{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}
]}}

MessageItem 流式传输设计

我们从模型输出里拿到的原始 message item 如下:

"content": [
{
"type": "text",
"text": "The weather in Paris is sunny with a temperature of 15C.[^1]",
}
]

对 text 进行解析,带上引用,转换为 block_list :

"block_list": [
{
"type": "text",
"text": "The weather in Paris is sunny with a temperature of 15C.[^1]",
"annotations": [
{
"type": "reference_to_block",
"reference_id": 1,
"start_index": 44,
"end_index": 47
},
]
},
]

将带引用的 block_list 进行流式传输:

// Added
{"type":"task.output_item.added","task_id":"task_1234xyz","output_index":3,"item":{"type":"message","id":"msg_1234xyz","role":"assistant","block_list":[]}}
// Deltas for block_list
{"type":"task.text.done","task_id":"task_1234xyz","item_id":"msg_1234xyz","output_index":3,"block_index":0,"item":{"type":"text","text":"The weather in Paris is sunny with a temperature of 15C.[^1]","id":1,"annotations":[{"type":"reference_to_block","reference_id":1,"start_index":44,"end_index":47}]}}
// Done
{"type":"task.output_item.done","task_id":"task_1234xyz","output_index":3,"item":{"type":"message","id":"msg_1234xyz","role":"assistant","block_list":[{"type":"text","text":"The weather in Paris is sunny with a temperature of 15C.[^1]","id":1,"annotations":[{"type":"reference_to_block","reference_id":1,"start_index":44,"end_index":47}]}]}}

子 agent 输出处理

在子 agent 作为工具调用的场景下,子 agent 的输出需要经过处理后嵌入到主 agent 的 tool_result 中。

  1. 只有 message 部分加上引用池会嵌入到主 agent的 tool_result 的 content 里
  2. 所有部分会转换为 block 嵌入到主 agent 的 tool_result 的 block_list 里。

我们考虑主 agent 调用子 agent 获取天气信息的场景。

假设主 agent 的输出如下:

{
"task_id": "task_1234xyz",
"output": [
{
"type": "reasoning",
"id": "rs_1234xyz",
"summary": [
{
"type": "text",
"text": "Thinking about the weather in Paris."
},
{
"type": "text",
"text": "Decided to call ask_for_help function."
}
],
},
{
"type": "tool_call",
"id": "fc_1234xyz",
"call_id": "call_1234xyz",
"name": "ask_for_help",
"arguments": "{\"name\":\"WeatherAgent\",\"description\":\"An agent that provides weather information.\",\"prompt\":\"Get the current weather in Paris, France.\"}",
},
{
"type": "tool_result",
"id": "fco_1234xyz",
"call_id": "call_1234xyz",
"content": [ // <- 主 agent tool_result 的 content 只包含子 agent 的 message 部分加上引用池
{
"type": "text",
"text": "<referencable-item>\nID: 1",
"id": 1,
"tags": ["added_by_reference_manager"],
}, // <- 添加引用池标记
{
"type": "text",
"text": "{\"temperature\":\"15C\",\"condition\":\"Sunny\"}",
"id": 1,
},
{
"type": "image_url",
"image_url": {
"url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
},
"id": 1,
},
{
"type": "text",
"text": "</referencable-item>",
"id": 1,
"tags": ["added_by_reference_manager"],
}, // <- 添加引用池标记
// ---- 以上是 message 中的引用池部分 ----
// 以下是子 agent 的 message 部分
{
"type": "text",
"text": "The weather in Paris is sunny with a temperature of 15C.[^1]",
},
],
"block_list": [ // <- 主 agent tool_result 的 block_list 包含子 agent 的所有输出部分, reasoning, tool_call, tool_result, message
{
"type": "reasoning",
"summary": [
{
"type": "text",
"text": "Thinking about the weather in Paris."
},
{
"type": "text",
"text": "Decided to call get_weather function."
}
],
},
{
"type": "tool_call",
"id": "fc_1234xyz",
"call_id": "call_1234xyz",
"name": "get_weather",
"arguments": "{\"location\":\"Paris, France\"}",
},
{
"type": "tool_result",
"id": "fco_1234xyz",
"call_id": "call_1234xyz",
"block_list": [
{
"type": "text",
"text": "The current weather in Paris is sunny with a temperature of 15C.",
"id": 1,
},
{
"type": "image_url",
"image_url": {
"url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
},
"id": 1,
}
]
},
{
"type": "message",
"id": "msg_1234xyz",
"role": "assistant",
"block_list": [
{
"type": "text",
"text": "The weather in Paris is sunny with a temperature of 15C.[^1]",
"id": 1,
"annotations": [
{
"type": "reference_to_block",
"reference_id": 1,
"start_index": 44,
"end_index": 47
},
]
},
]
},
]
},
{
"type": "message",
"id": "msg_1234xyz",
"role": "assistant",
"content": [
{
"type": "text",
"text": "The weather in Paris is sunny with a temperature of 15C.[^1]",
}
],
"block_list": [
{
"type": "text",
"text": "The weather in Paris is sunny with a temperature of 15C.[^1]",
"id": 1,
"annotations": [
{
"type": "reference_to_block",
"reference_id": 1,
"start_index": 44,
"end_index": 47
},
]
},
]
},
]
}

调用 ask_for_help 函数,获得 WeatherAgent 的输出如下:

{
"task_id": "call_1234xyz", //<- 子 agent 的 task_id 是主 agent tool_call 的 call_id
"output": [
{
"type": "reasoning",
"id": "rs_1234xyz",
"summary": [
{
"type": "text",
"text": "Thinking about the weather in Paris."
},
{
"type": "text",
"text": "Decided to call get_weather function."
}
],
},
{
"type": "tool_call",
"id": "fc_1234xyz",
"call_id": "call_1234xyz",
"name": "get_weather",
"arguments": "{\"location\":\"Paris, France\"}",
},
{
"type": "tool_result",
"id": "fco_1234xyz",
"call_id": "call_1234xyz",
"content": [
{
"type": "text",
"text": "<referencable-item>\nID: 1",
"id": 1,
"tags": ["added_by_reference_manager"],
}, // <- 添加引用池标记
{
"type": "text",
"text": "{\"temperature\":\"15C\",\"condition\":\"Sunny\"}",
"id": 1,
},
{
"type": "text",
"text": "</referencable-item>",
"id": 1,
"tags": ["added_by_reference_manager"],
} // <- 添加引用池标记
],
"block_list": [
{
"type": "text",
"text": "The current weather in Paris is sunny with a temperature of 15C.",
"id": 1,
},
{
"type": "image_url",
"image_url": {
"url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
},
"id": 1,
}
]
},
{
"type": "message",
"id": "msg_1234xyz",
"role": "assistant",
"content": [
{
"type": "text",
"text": "The weather in Paris is sunny with a temperature of 15C.[^1]",
}
],
"block_list": [
{
"type": "text",
"text": "The weather in Paris is sunny with a temperature of 15C.[^1]",
"id": 1,
"annotations": [
{
"type": "reference_to_block",
"reference_id": 1,
"start_index": 44,
"end_index": 47
},
]
},
]
},
]
}

可以看到,子 agent 的输出就是正常的 response 结构。

子 agent 的 event stream 如下:

// Reasoning item
// Added
{"type":"task.output_item.added","task_id":"call_1234xyz","output_index":0,"item":{"type":"reasoning","id":"rs_1234xyz","summary":[]}}
// First summary item
{"type":"task.reasoning_summary_item.added","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"item":{"type":"text","text":""}}
{"type":"task.reasoning_summary_text.delta","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"delta":"Thinking about the weather "}
{"type":"task.reasoning_summary_text.delta","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"delta":"in Paris."}
{"type":"task.reasoning_summary_item.done","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"item":{"type":"text","text":"Thinking about the weather in Paris."}}
// Second summary item
{"type":"task.reasoning_summary_item.added","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"item":{"type":"text","text":""}}
{"type":"task.reasoning_summary_text.delta","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"delta":"Decided to call "}
{"type":"task.reasoning_summary_text.delta","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"delta":"get_weather function."}
{"type":"task.reasoning_summary_item.done","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"item":{"type":"text","text":"Decided to call get_weather function."}}
// Done
{"type":"task.output_item.done","task_id":"call_1234xyz","output_index":0,"item":{"type":"reasoning","id":"rs_1234xyz","summary":[
{"type":"text","text":"Thinking about the weather in Paris."},
{"type":"text","text":"Decided to call get_weather function."}
]}}

// Function call item
// Added
{"type":"task.output_item.added","task_id":"call_1234xyz","output_index":1,"item":{"type":"tool_call","id":"fc_1234xyz","call_id":"call_1234xyz","name":"get_weather","arguments":""}}
// Deltas for arguments
{"type":"task.tool_call_arguments.delta","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"{\""}
{"type":"task.tool_call_arguments.delta","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"location"}
{"type":"task.tool_call_arguments.delta","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"\":\""}
{"type":"task.tool_call_arguments.delta","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"Paris"}
{"type":"task.tool_call_arguments.delta","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":","}
{"type":"task.tool_call_arguments.delta","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":" France"}
{"type":"task.tool_call_arguments.delta","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"\"}"}
{"type":"task.tool_call_arguments.done","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"arguments":"{\"location\":\"Paris, France\"}"}
// Done
{"type":"task.output_item.done","task_id":"call_1234xyz","output_index":1,"item":{"type":"tool_call","id":"fc_1234xyz","call_id":"call_1234xyz","name":"get_weather","arguments":"{\"location\":\"Paris, France\"}"}}

// Function call output item
// Added: 初始状态下 content 和 block_list 都是空的
{"type":"task.output_item.added","task_id":"call_1234xyz","output_index":2,"item":{"type":"tool_result","id":"fco_1234xyz","call_id":"call_1234xyz","block_list":[]}}
// Deltas for block_list
// 1. 对于较短的 block,一次性传输
{"type":"task.text.done","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":0,"item":{"type":"text","text":"{\"temperature\":\"15C\",\"condition\":\"Sunny\"}","id":1}}
// 2. 对于较大的 block,带上状态进行传输
{"type":"task.image.added","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":""},"id":1}}
{"type":"task.image.done","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
// 3. 对于超大的 block,可以继续拆分 delta 进行传输
// image - added 前端开始准备传输
{"type":"task.image.added","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":""},"id":1}}
// image - deltas 传输中,每个 delta 都是一个完整的图片
{"type":"task.image.delta","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"partial_image_index":0,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
{"type":"task.image.delta","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"partial_image_index":1,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
// image - done 传输完成,最后一个图片才是最终状态的图片
{"type":"task.image.done","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
// Done
{"type":"task.output_item.done","task_id":"call_1234xyz","output_index":2,"item":{"type":"tool_result","id":"fco_1234xyz","call_id":"call_1234xyz","block_list":[
{"type":"text","text":"{\"temperature\":\"15C\",\"condition\":\"Sunny\"}","id":1},
{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}
]}}

// Message item
// Added
{"type":"task.output_item.added","task_id":"call_1234xyz","output_index":3,"item":{"type":"message","id":"msg_1234xyz","role":"assistant","block_list":[]}}
// Deltas for block_list
{"type":"task.text.done","task_id":"call_1234xyz","item_id":"msg_1234xyz","output_index":3,"block_index":0,"item":{"type":"text","text":"The weather in Paris is sunny with a temperature of 15C.[^1]","id":1,"annotations":[{"type":"reference_to_block","reference_id":1,"start_index":44,"end_index":47}]}}
// Done
{"type":"task.output_item.done","task_id":"call_1234xyz","output_index":3,"item":{"type":"message","id":"msg_1234xyz","role":"assistant","block_list":[{"type":"text","text":"The weather in Paris is sunny with a temperature of 15C.[^1]","id":1,"annotations":[{"type":"reference_to_block","reference_id":1,"start_index":44,"end_index":47}]}]}}

接下来考虑把子 agent 的 event stream 嵌入到主 agent 的 event stream 里。

// Added: 主 agent 创建了一个承接 call_id 输出的容器
{"type":"task.output_item.added","task_id":"task_1234xyz","output_index":2,"item":{"type":"tool_result","id":"fco_1234xyz","call_id":"call_1234xyz","block_list":[]}}
// Deltas for block_list
// --- 大部分情况下可以直接透传子 agent 的 event stream 给前端。
// --- 前端自动根据 task_id 将对应的流放到正确的容器里
// --- 以下直接透传
// Reasoning item
// Added
{"type":"task.output_item.added","task_id":"call_1234xyz","output_index":0,"item":{"type":"reasoning","id":"rs_1234xyz","summary":[]}}
// First summary item
{"type":"task.reasoning_summary_item.added","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"item":{"type":"text","text":""}}
{"type":"task.reasoning_summary_text.delta","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"delta":"Thinking about the weather "}
{"type":"task.reasoning_summary_text.delta","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"delta":"in Paris."}
{"type":"task.reasoning_summary_item.done","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"item":{"type":"text","text":"Thinking about the weather in Paris."}}
// Second summary item
{"type":"task.reasoning_summary_item.added","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"item":{"type":"text","text":""}}
{"type":"task.reasoning_summary_text.delta","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"delta":"Decided to call "}
{"type":"task.reasoning_summary_text.delta","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"delta":"get_weather function."}
{"type":"task.reasoning_summary_item.done","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"item":{"type":"text","text":"Decided to call get_weather function."}}
// Done
{"type":"task.output_item.done","task_id":"call_1234xyz","output_index":0,"item":{"type":"reasoning","id":"rs_1234xyz","summary":[
{"type":"text","text":"Thinking about the weather in Paris."},
{"type":"text","text":"Decided to call get_weather function."}
]}}

// Function call item
// Added
{"type":"task.output_item.added","task_id":"call_1234xyz","output_index":1,"item":{"type":"tool_call","id":"fc_1234xyz","call_id":"call_1234xyz","name":"get_weather","arguments":""}}
// Deltas for arguments
{"type":"task.tool_call_arguments.delta","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"{\"location\":\"Paris"}
{"type":"task.tool_call_arguments.delta","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":", France\"}"}
{"type":"task.tool_call_arguments.done","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"arguments":"{\"location\":\"Paris, France\"}"}
// Done
{"type":"task.output_item.done","task_id":"call_1234xyz","output_index":1,"item":{"type":"tool_call","id":"fc_1234xyz","call_id":"call_1234xyz","name":"get_weather","arguments":"{\"location\":\"Paris, France\"}"}}

// Function call output item
// Added: 初始状态下 content 和 block_list 都是空的
{"type":"task.output_item.added","task_id":"call_1234xyz","output_index":2,"item":{"type":"tool_result","id":"fco_1234xyz","call_id":"call_1234xyz","block_list":[]}}
// Deltas for block_list
// 1. 对于较短的 block,一次性传输
{"type":"task.text.done","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":0,"item":{"type":"text","text":"{\"temperature\":\"15C\",\"condition\":\"Sunny\"}","id":1}}
// 2. 对于较大的 block,带上状态进行传输
{"type":"task.image.added","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":""},"id":1}}
{"type":"task.image.done","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
// 3. 对于超大的 block,可以继续拆分 delta 进行传输
// image - added 前端开始准备传输
{"type":"task.image.added","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":""},"id":1}}
// image - deltas 传输中,每个 delta 都是一个完整的图片
{"type":"task.image.delta","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"partial_image_index":0,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
{"type":"task.image.delta","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"partial_image_index":1,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
// image - done 传输完成,最后一个图片才是最终状态的图片
{"type":"task.image.done","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
// Done
{"type":"task.output_item.done","task_id":"call_1234xyz","output_index":2,"item":{"type":"tool_result","id":"fco_1234xyz","call_id":"call_1234xyz","block_list":[
{"type":"text","text":"{\"temperature\":\"15C\",\"condition\":\"Sunny\"}","id":1},
{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}
]}}

// Message item
// Added
{"type":"task.output_item.added","task_id":"call_1234xyz","output_index":3,"item":{"type":"message","id":"msg_1234xyz","role":"assistant","block_list":[]}}
// Deltas for block_list
{"type":"task.text.done","task_id":"call_1234xyz","item_id":"msg_1234xyz","output_index":3,"block_index":0,"item":{"type":"text","text":"The weather in Paris is sunny with a temperature of 15C.[^1]","id":1,"annotations":[{"type":"reference_to_block","reference_id":1,"start_index":44,"end_index":47}]}}
// Done
{"type":"task.output_item.done","task_id":"call_1234xyz","output_index":3,"item":{"type":"message","id":"msg_1234xyz","role":"assistant","block_list":[{"type":"text","text":"The weather in Paris is sunny with a temperature of 15C.[^1]","id":1,"annotations":[{"type":"reference_to_block","reference_id":1,"start_index":44,"end_index":47}]}]}}
// --- 以上是子 agent 的 event stream 透传结束 ---
// 此时,前端收集到的主 agent tool_result 的 block_list 已经包含了子 agent 的所有输出部分
// {"type":"tool_result","id":"fco_1234xyz","task_id":"task_1234xyz","call_id":"call_1234xyz","block_list":[
// {"type":"reasoning","summary":[
// {"type":"text","text":"Thinking about the weather in Paris."},
// {"type":"text","text":"Decided to call get_weather function."}
// ]},
// {"type":"tool_call","id":"fc_1234xyz","task_id":"task_1234xyz","call_id":"call_1234xyz","name":"get_weather","arguments":"{\"location\":\"Paris, France\"}"},
// {"type":"tool_result","id":"fco_1234xyz","task_id":"task_1234xyz","call_id":"call_1234xyz","block_list":[
// {"type":"text","text":"The current weather in Paris is sunny with a temperature of 15C.","id":1},
// {"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}
// ]},
// {"type":"message","id":"msg_1234xyz","task_id":"task_1234xyz","role":"assistant","block_list":[
// {"type":"text","text":"The weather in Paris is sunny with a temperature of 15C.[^1]","id":1,"annotations":[{"type":"reference_to_block","reference_id":1,"start_index":44,"end_index":47}]}
// ]}
// ]}
// Done: 主 agent 的 tool_result item 传输完成
{"type":"task.output_item.done","task_id":"task_1234xyz","output_index":2,"item":{"type":"tool_result","id":"fco_1234xyz","call_id":"call_1234xyz","status":"completed"}} // <- 注意,这里只传输状态完成,不需要重复传输 block_list