跳到主要内容

1 篇博文 含有标签「Design」

Design documents and discussions about AgentLin's architecture and features.

查看所有标签

如何设计一个有状态的流式响应系统

· 阅读需 18 分钟
Xueyuan Lin
Xueyuan Lin
Agentlin maintainer

在构建对话式AI系统时，设计一个能够处理有状态流式响应的系统是至关重要的。本文将探讨如何设计一个有状态的流式响应系统，以提升用户体验和系统性能。

流式传输​

首先确定系统的最终状态是输出一个结构化的大 json 对象。 其次，系统应该以流式方式逐步组装出这个大对象，在每一步都能向用户展示当前的进展和进度状态。

终点状态​

系统的最终状态是一个完整的 json 对象。我们考虑 4 种消息类型：reasoning，tool_call，tool_result，message。

{
  "task_id": "task_1234xyz",
  "output": [
    {
      "type": "reasoning",
      "id": "rs_1234xyz",
      "summary": [
        {
          "type": "text",
          "text": "Thinking about the weather in Paris."
        },
        {
          "type": "text",
          "text": "Decided to call get_weather function."
        }
      ],
    },
    {
      "type": "tool_call",
      "id": "fc_1234xyz",
      "call_id": "call_1234xyz",
      "name": "get_weather",
      "arguments": "{\"location\":\"Paris, France\"}",
    },
    {
      "type": "tool_result",
      "id": "fco_1234xyz",
      "call_id": "call_1234xyz",
      "content": [
        {
          "type": "text",
          "text": "<referencable-item>\nID: 1",
          "id": 1,
          "tags": ["added_by_reference_manager"],
        },  // <- 添加引用池标记
        {
          "type": "text",
          "text": "{\"temperature\":\"15C\",\"condition\":\"Sunny\"}",
          "id": 1,
        },
        {
          "type": "image_url",
          "image_url": {
            "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
          },
          "id": 1,
        },
        {
          "type": "text",
          "text": "</referencable-item>",
          "id": 1,
          "tags": ["added_by_reference_manager"],
        }  // <- 添加引用池标记
      ],
      "block_list": [
        {
          "type": "text",
          "text": "{\"temperature\":\"15C\",\"condition\":\"Sunny\"}",
          "id": 1,
        },
        {
          "type": "image_url",
          "image_url": {
            "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
          },
          "id": 1,
        }
      ]
    },
    {
      "type": "message",
      "id": "msg_1234xyz",
      "role": "assistant",
      "content": [
        {
          "type": "text",
          "text": "The weather in Paris is sunny with a temperature of 15C.[^1]",
        }
      ],
      "block_list": [
        {
          "type": "text",
          "text": "The weather in Paris is sunny with a temperature of 15C.[^1]",
          "id": 1,
          "annotations": [
            {
              "type": "reference_to_block",
              "reference_id": 1,
              "start_index": 44,
              "end_index": 47
            },
          ]
        },
      ]
    },
  ]
}

子 agent 作为工具的场景下，子 agent 的输出也对应一个 task_id，但会经过处理：

  1. 只有 message 部分加上引用池会嵌入到主 agent 的 tool_result 的 content 里
  2. 所有部分会转换为 block 嵌入到主 agent 的 tool_result 的 block_list 里。 所以主 agent 只看到子 agent的结果 message，而用户能看到子 agent 的包括思考和工具调用的调用过程。

event 设计​

首先，所有的 event 都是围绕 task_id 进行的；所有的 event 都有严格的类型定义。所以，所有的 event 都有两个核心字段：type 和 task_id。

根据 event 的类型不同，设计不同的 event 结构。

我们主要面向 agent 的输出，而 agent 的输出包含 4 中类型的 output item：reasoning，tool_call，tool_result，message。 因此，我们设计 4 种 output item 的流式传输方式。

ReasoningItem 流式传输设计​

考虑以下 reasoning item：

{
  "type": "reasoning",
  "id": "rs_1234xyz",
  "summary": [
    {
      "type": "text",
      "text": "Thinking about the weather in Paris."
    },
    {
      "type": "text",
      "text": "Decided to call get_weather function."
    }
  ],
}

对于 summary 字段，由于是一个列表，我们可以将列表拆分成多个 item 进行流式传输。每个 item 内部如果有长文本，也可以继续拆分成 delta 进行流式传输。

// Added
{"type":"task.output_item.added","task_id":"task_1234xyz","output_index":0,"item":{"type":"reasoning","id":"rs_1234xyz","summary":[]}}
// First summary item
{"type":"task.reasoning_summary_item.added","task_id":"task_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"item":{"type":"text","text":""}}
{"type":"task.reasoning_summary_text.delta","task_id":"task_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"delta":"Thinking about the weather "}
{"type":"task.reasoning_summary_text.delta","task_id":"task_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"delta":"in Paris."}
{"type":"task.reasoning_summary_item.done","task_id":"task_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"item":{"type":"text","text":"Thinking about the weather in Paris."}}
// Second summary item
{"type":"task.reasoning_summary_item.added","task_id":"task_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"item":{"type":"text","text":""}}
{"type":"task.reasoning_summary_text.delta","task_id":"task_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"delta":"Decided to call "}
{"type":"task.reasoning_summary_text.delta","task_id":"task_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"delta":"get_weather function."}
{"type":"task.reasoning_summary_item.done","task_id":"task_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"item":{"type":"text","text":"Decided to call get_weather function."}}
// Done
{"type":"task.output_item.done","task_id":"task_1234xyz","output_index":0,"item":{"type":"reasoning","id":"rs_1234xyz","summary":[
  {"type":"text","text":"Thinking about the weather in Paris."},
  {"type":"text","text":"Decided to call get_weather function."}
]}}

ToolCallItem 流式传输设计​

考虑以下 tool_call item：

{
  "type": "tool_call",
  "id": "fc_1234xyz",
  "call_id": "call_1234xyz",
  "name": "get_weather",
  "arguments": "{\"location\":\"Paris, France\"}",
}

对于短的文本（如 name），可以直接传，不需要拆分成 delta。对于长文本（如 arguments），需要拆分成 delta 进行流式传输。

// Added
{"type":"task.output_item.added","task_id":"task_1234xyz","output_index":1,"item":{"type":"tool_call","id":"fc_1234xyz","call_id":"call_1234xyz","name":"get_weather","arguments":""}}
// Deltas for arguments
{"type":"task.tool_call_arguments.delta","task_id":"task_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"{\""}
{"type":"task.tool_call_arguments.delta","task_id":"task_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"location"}
{"type":"task.tool_call_arguments.delta","task_id":"task_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"\":\""}
{"type":"task.tool_call_arguments.delta","task_id":"task_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"Paris"}
{"type":"task.tool_call_arguments.delta","task_id":"task_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":","}
{"type":"task.tool_call_arguments.delta","task_id":"task_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":" France"}
{"type":"task.tool_call_arguments.delta","task_id":"task_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"\"}"}
{"type":"task.tool_call_arguments.done","task_id":"task_1234xyz","item_id":"fc_1234xyz","output_index":1,"arguments":"{\"location\":\"Paris, France\"}"}
// Done
{"type":"task.output_item.done","task_id":"task_1234xyz","output_index":1,"item":{"type":"tool_call","id":"fc_1234xyz","call_id":"call_1234xyz","name":"get_weather","arguments":"{\"location\":\"Paris, France\"}"}}

ToolResultItem 流式传输设计​

我们假设工具的输出为一个包含 message_content 和 block_list 的结构化对象。

{
  "message_content": [
    {
      "type": "text",
      "text": "{\"temperature\":\"15C\",\"condition\":\"Sunny\"}",
      "id": 1,
    },
    {
      "type": "image_url",
      "image_url": {
        "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
      },
      "id": 1,
    },
  ],
  "block_list": [
    {
      "type": "text",
      "text": "{\"temperature\":\"15C\",\"condition\":\"Sunny\"}",
      "id": 1,
    },
    {
      "type": "image_url",
      "image_url": {
        "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
      },
      "id": 1,
    }
  ]
}

根据引用协议，将会被处理为以下 tool_result item：

{
  "type": "tool_result",
  "id": "fco_1234xyz",
  "call_id": "call_1234xyz",
  "content": [
    {
      "type": "text",
      "text": "<referencable-item>\nID: 1",
      "id": 1,
      "tags": ["added_by_reference_manager"],
    },  // <- 添加引用池标记
    {
      "type": "text",
      "text": "{\"temperature\":\"15C\",\"condition\":\"Sunny\"}",
      "id": 1,
    },
    {
      "type": "image_url",
      "image_url": {
        "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
      },
      "id": 1,
    },
    {
      "type": "text",
      "text": "</referencable-item>",
      "id": 1,
      "tags": ["added_by_reference_manager"],
    }  // <- 添加引用池标记
  ],
  "block_list": [
    {
      "type": "text",
      "text": "{\"temperature\":\"15C\",\"condition\":\"Sunny\"}",
      "id": 1,
    },
    {
      "type": "image_url",
      "image_url": {
        "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
      },
      "id": 1,
    }
  ]
}

给前端只传 block_list，所以只需要将 block_list 处理为以下流式传输事件：

// Added: 初始状态下 content 和 block_list 都是空的
{"type":"task.output_item.added","task_id":"task_1234xyz","output_index":2,"item":{"type":"tool_result","id":"fco_1234xyz","call_id":"call_1234xyz","block_list":[]}}
// Deltas for block_list
// 1. 对于较短的 block，一次性传输
{"type":"task.text.done","task_id":"task_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":0,"item":{"type":"text","text":"{\"temperature\":\"15C\",\"condition\":\"Sunny\"}","id":1}}
// 2. 对于较大的 block，带上状态进行传输
{"type":"task.image.added","task_id":"task_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":""},"id":1}}
{"type":"task.image.done","task_id":"task_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
// 3. 对于超大的 block，可以继续拆分 delta 进行传输
// image - added 前端开始准备传输
{"type":"task.image.added","task_id":"task_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":""},"id":1}}
// image - deltas 传输中，每个 delta 都是一个完整的图片
{"type":"task.image.delta","task_id":"task_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"partial_image_index":0,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
{"type":"task.image.delta","task_id":"task_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"partial_image_index":1,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
// image - done 传输完成，最后一个图片才是最终状态的图片
{"type":"task.image.done","task_id":"task_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
// Done
{"type":"task.output_item.done","task_id":"task_1234xyz","output_index":2,"item":{"type":"tool_result","id":"fco_1234xyz","call_id":"call_1234xyz","block_list":[
  {"type":"text","text":"{\"temperature\":\"15C\",\"condition\":\"Sunny\"}","id":1},
  {"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}
]}}

MessageItem 流式传输设计​

我们从模型输出里拿到的原始 message item 如下：

"content": [
  {
    "type": "text",
    "text": "The weather in Paris is sunny with a temperature of 15C.[^1]",
  }
]

对 text 进行解析，带上引用，转换为 block_list ：

"block_list": [
  {
    "type": "text",
    "text": "The weather in Paris is sunny with a temperature of 15C.[^1]",
    "annotations": [
      {
        "type": "reference_to_block",
        "reference_id": 1,
        "start_index": 44,
        "end_index": 47
      },
    ]
  },
]

将带引用的 block_list 进行流式传输：

// Added
{"type":"task.output_item.added","task_id":"task_1234xyz","output_index":3,"item":{"type":"message","id":"msg_1234xyz","role":"assistant","block_list":[]}}
// Deltas for block_list
{"type":"task.text.done","task_id":"task_1234xyz","item_id":"msg_1234xyz","output_index":3,"block_index":0,"item":{"type":"text","text":"The weather in Paris is sunny with a temperature of 15C.[^1]","id":1,"annotations":[{"type":"reference_to_block","reference_id":1,"start_index":44,"end_index":47}]}}
// Done
{"type":"task.output_item.done","task_id":"task_1234xyz","output_index":3,"item":{"type":"message","id":"msg_1234xyz","role":"assistant","block_list":[{"type":"text","text":"The weather in Paris is sunny with a temperature of 15C.[^1]","id":1,"annotations":[{"type":"reference_to_block","reference_id":1,"start_index":44,"end_index":47}]}]}}

子 agent 输出处理​

在子 agent 作为工具调用的场景下，子 agent 的输出需要经过处理后嵌入到主 agent 的 tool_result 中。

  1. 只有 message 部分加上引用池会嵌入到主 agent的 tool_result 的 content 里
  2. 所有部分会转换为 block 嵌入到主 agent 的 tool_result 的 block_list 里。

我们考虑主 agent 调用子 agent 获取天气信息的场景。

假设主 agent 的输出如下：

{
  "task_id": "task_1234xyz",
  "output": [
    {
      "type": "reasoning",
      "id": "rs_1234xyz",
      "summary": [
        {
          "type": "text",
          "text": "Thinking about the weather in Paris."
        },
        {
          "type": "text",
          "text": "Decided to call ask_for_help function."
        }
      ],
    },
    {
      "type": "tool_call",
      "id": "fc_1234xyz",
      "call_id": "call_1234xyz",
      "name": "ask_for_help",
      "arguments": "{\"name\":\"WeatherAgent\",\"description\":\"An agent that provides weather information.\",\"prompt\":\"Get the current weather in Paris, France.\"}",
    },
    {
      "type": "tool_result",
      "id": "fco_1234xyz",
      "call_id": "call_1234xyz",
      "content": [  // <- 主 agent tool_result 的 content 只包含子 agent 的 message 部分加上引用池
        {
          "type": "text",
          "text": "<referencable-item>\nID: 1",
          "id": 1,
          "tags": ["added_by_reference_manager"],
        },  // <- 添加引用池标记
        {
          "type": "text",
          "text": "{\"temperature\":\"15C\",\"condition\":\"Sunny\"}",
          "id": 1,
        },
        {
          "type": "image_url",
          "image_url": {
            "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
          },
          "id": 1,
        },
        {
          "type": "text",
          "text": "</referencable-item>",
          "id": 1,
          "tags": ["added_by_reference_manager"],
        },  // <- 添加引用池标记
        // ---- 以上是 message 中的引用池部分 ----
        // 以下是子 agent 的 message 部分
        {
          "type": "text",
          "text": "The weather in Paris is sunny with a temperature of 15C.[^1]",
        },
      ],
      "block_list": [  // <- 主 agent tool_result 的 block_list 包含子 agent 的所有输出部分, reasoning, tool_call, tool_result, message
        {
          "type": "reasoning",
          "summary": [
            {
              "type": "text",
              "text": "Thinking about the weather in Paris."
            },
            {
              "type": "text",
              "text": "Decided to call get_weather function."
            }
          ],
        },
        {
          "type": "tool_call",
          "id": "fc_1234xyz",
          "call_id": "call_1234xyz",
          "name": "get_weather",
          "arguments": "{\"location\":\"Paris, France\"}",
        },
        {
          "type": "tool_result",
          "id": "fco_1234xyz",
          "call_id": "call_1234xyz",
          "block_list": [
            {
              "type": "text",
              "text": "The current weather in Paris is sunny with a temperature of 15C.",
              "id": 1,
            },
            {
              "type": "image_url",
              "image_url": {
                "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
              },
              "id": 1,
            }
          ]
        },
        {
          "type": "message",
          "id": "msg_1234xyz",
          "role": "assistant",
          "block_list": [
            {
              "type": "text",
              "text": "The weather in Paris is sunny with a temperature of 15C.[^1]",
              "id": 1,
              "annotations": [
                {
                  "type": "reference_to_block",
                  "reference_id": 1,
                  "start_index": 44,
                  "end_index": 47
                },
              ]
            },
          ]
        },
      ]
    },
    {
      "type": "message",
      "id": "msg_1234xyz",
      "role": "assistant",
      "content": [
        {
          "type": "text",
          "text": "The weather in Paris is sunny with a temperature of 15C.[^1]",
        }
      ],
      "block_list": [
        {
          "type": "text",
          "text": "The weather in Paris is sunny with a temperature of 15C.[^1]",
          "id": 1,
          "annotations": [
            {
              "type": "reference_to_block",
              "reference_id": 1,
              "start_index": 44,
              "end_index": 47
            },
          ]
        },
      ]
    },
  ]
}

调用 ask_for_help 函数，获得 WeatherAgent 的输出如下：

{
  "task_id": "call_1234xyz",  //<- 子 agent 的 task_id 是主 agent tool_call 的 call_id
  "output": [
    {
      "type": "reasoning",
      "id": "rs_1234xyz",
      "summary": [
        {
          "type": "text",
          "text": "Thinking about the weather in Paris."
        },
        {
          "type": "text",
          "text": "Decided to call get_weather function."
        }
      ],
    },
    {
      "type": "tool_call",
      "id": "fc_1234xyz",
      "call_id": "call_1234xyz",
      "name": "get_weather",
      "arguments": "{\"location\":\"Paris, France\"}",
    },
    {
      "type": "tool_result",
      "id": "fco_1234xyz",
      "call_id": "call_1234xyz",
      "content": [
        {
          "type": "text",
          "text": "<referencable-item>\nID: 1",
          "id": 1,
          "tags": ["added_by_reference_manager"],
        },  // <- 添加引用池标记
        {
          "type": "text",
          "text": "{\"temperature\":\"15C\",\"condition\":\"Sunny\"}",
          "id": 1,
        },
        {
          "type": "text",
          "text": "</referencable-item>",
          "id": 1,
          "tags": ["added_by_reference_manager"],
        }  // <- 添加引用池标记
      ],
      "block_list": [
        {
          "type": "text",
          "text": "The current weather in Paris is sunny with a temperature of 15C.",
          "id": 1,
        },
        {
          "type": "image_url",
          "image_url": {
            "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
          },
          "id": 1,
        }
      ]
    },
    {
      "type": "message",
      "id": "msg_1234xyz",
      "role": "assistant",
      "content": [
        {
          "type": "text",
          "text": "The weather in Paris is sunny with a temperature of 15C.[^1]",
        }
      ],
      "block_list": [
        {
          "type": "text",
          "text": "The weather in Paris is sunny with a temperature of 15C.[^1]",
          "id": 1,
          "annotations": [
            {
              "type": "reference_to_block",
              "reference_id": 1,
              "start_index": 44,
              "end_index": 47
            },
          ]
        },
      ]
    },
  ]
}

可以看到，子 agent 的输出就是正常的 response 结构。

子 agent 的 event stream 如下：

// Reasoning item
// Added
{"type":"task.output_item.added","task_id":"call_1234xyz","output_index":0,"item":{"type":"reasoning","id":"rs_1234xyz","summary":[]}}
// First summary item
{"type":"task.reasoning_summary_item.added","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"item":{"type":"text","text":""}}
{"type":"task.reasoning_summary_text.delta","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"delta":"Thinking about the weather "}
{"type":"task.reasoning_summary_text.delta","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"delta":"in Paris."}
{"type":"task.reasoning_summary_item.done","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"item":{"type":"text","text":"Thinking about the weather in Paris."}}
// Second summary item
{"type":"task.reasoning_summary_item.added","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"item":{"type":"text","text":""}}
{"type":"task.reasoning_summary_text.delta","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"delta":"Decided to call "}
{"type":"task.reasoning_summary_text.delta","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"delta":"get_weather function."}
{"type":"task.reasoning_summary_item.done","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"item":{"type":"text","text":"Decided to call get_weather function."}}
// Done
{"type":"task.output_item.done","task_id":"call_1234xyz","output_index":0,"item":{"type":"reasoning","id":"rs_1234xyz","summary":[
  {"type":"text","text":"Thinking about the weather in Paris."},
  {"type":"text","text":"Decided to call get_weather function."}
]}}

// Function call item
// Added
{"type":"task.output_item.added","task_id":"call_1234xyz","output_index":1,"item":{"type":"tool_call","id":"fc_1234xyz","call_id":"call_1234xyz","name":"get_weather","arguments":""}}
// Deltas for arguments
{"type":"task.tool_call_arguments.delta","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"{\""}
{"type":"task.tool_call_arguments.delta","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"location"}
{"type":"task.tool_call_arguments.delta","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"\":\""}
{"type":"task.tool_call_arguments.delta","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"Paris"}
{"type":"task.tool_call_arguments.delta","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":","}
{"type":"task.tool_call_arguments.delta","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":" France"}
{"type":"task.tool_call_arguments.delta","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"\"}"}
{"type":"task.tool_call_arguments.done","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"arguments":"{\"location\":\"Paris, France\"}"}
// Done
{"type":"task.output_item.done","task_id":"call_1234xyz","output_index":1,"item":{"type":"tool_call","id":"fc_1234xyz","call_id":"call_1234xyz","name":"get_weather","arguments":"{\"location\":\"Paris, France\"}"}}

// Function call output item
// Added: 初始状态下 content 和 block_list 都是空的
{"type":"task.output_item.added","task_id":"call_1234xyz","output_index":2,"item":{"type":"tool_result","id":"fco_1234xyz","call_id":"call_1234xyz","block_list":[]}}
// Deltas for block_list
// 1. 对于较短的 block，一次性传输
{"type":"task.text.done","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":0,"item":{"type":"text","text":"{\"temperature\":\"15C\",\"condition\":\"Sunny\"}","id":1}}
// 2. 对于较大的 block，带上状态进行传输
{"type":"task.image.added","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":""},"id":1}}
{"type":"task.image.done","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
// 3. 对于超大的 block，可以继续拆分 delta 进行传输
// image - added 前端开始准备传输
{"type":"task.image.added","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":""},"id":1}}
// image - deltas 传输中，每个 delta 都是一个完整的图片
{"type":"task.image.delta","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"partial_image_index":0,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
{"type":"task.image.delta","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"partial_image_index":1,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
// image - done 传输完成，最后一个图片才是最终状态的图片
{"type":"task.image.done","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
// Done
{"type":"task.output_item.done","task_id":"call_1234xyz","output_index":2,"item":{"type":"tool_result","id":"fco_1234xyz","call_id":"call_1234xyz","block_list":[
  {"type":"text","text":"{\"temperature\":\"15C\",\"condition\":\"Sunny\"}","id":1},
  {"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}
]}}

// Message item
// Added
{"type":"task.output_item.added","task_id":"call_1234xyz","output_index":3,"item":{"type":"message","id":"msg_1234xyz","role":"assistant","block_list":[]}}
// Deltas for block_list
{"type":"task.text.done","task_id":"call_1234xyz","item_id":"msg_1234xyz","output_index":3,"block_index":0,"item":{"type":"text","text":"The weather in Paris is sunny with a temperature of 15C.[^1]","id":1,"annotations":[{"type":"reference_to_block","reference_id":1,"start_index":44,"end_index":47}]}}
// Done
{"type":"task.output_item.done","task_id":"call_1234xyz","output_index":3,"item":{"type":"message","id":"msg_1234xyz","role":"assistant","block_list":[{"type":"text","text":"The weather in Paris is sunny with a temperature of 15C.[^1]","id":1,"annotations":[{"type":"reference_to_block","reference_id":1,"start_index":44,"end_index":47}]}]}}

接下来考虑把子 agent 的 event stream 嵌入到主 agent 的 event stream 里。

// Added: 主 agent 创建了一个承接 call_id 输出的容器
{"type":"task.output_item.added","task_id":"task_1234xyz","output_index":2,"item":{"type":"tool_result","id":"fco_1234xyz","call_id":"call_1234xyz","block_list":[]}}
// Deltas for block_list
// --- 大部分情况下可以直接透传子 agent 的 event stream 给前端。
// --- 前端自动根据 task_id 将对应的流放到正确的容器里
// --- 以下直接透传
// Reasoning item
// Added
{"type":"task.output_item.added","task_id":"call_1234xyz","output_index":0,"item":{"type":"reasoning","id":"rs_1234xyz","summary":[]}}
// First summary item
{"type":"task.reasoning_summary_item.added","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"item":{"type":"text","text":""}}
{"type":"task.reasoning_summary_text.delta","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"delta":"Thinking about the weather "}
{"type":"task.reasoning_summary_text.delta","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"delta":"in Paris."}
{"type":"task.reasoning_summary_item.done","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":0,"item":{"type":"text","text":"Thinking about the weather in Paris."}}
// Second summary item
{"type":"task.reasoning_summary_item.added","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"item":{"type":"text","text":""}}
{"type":"task.reasoning_summary_text.delta","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"delta":"Decided to call "}
{"type":"task.reasoning_summary_text.delta","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"delta":"get_weather function."}
{"type":"task.reasoning_summary_item.done","task_id":"call_1234xyz","item_id":"rs_1234xyz","output_index":0,"summary_index":1,"item":{"type":"text","text":"Decided to call get_weather function."}}
// Done
{"type":"task.output_item.done","task_id":"call_1234xyz","output_index":0,"item":{"type":"reasoning","id":"rs_1234xyz","summary":[
  {"type":"text","text":"Thinking about the weather in Paris."},
  {"type":"text","text":"Decided to call get_weather function."}
]}}

// Function call item
// Added
{"type":"task.output_item.added","task_id":"call_1234xyz","output_index":1,"item":{"type":"tool_call","id":"fc_1234xyz","call_id":"call_1234xyz","name":"get_weather","arguments":""}}
// Deltas for arguments
{"type":"task.tool_call_arguments.delta","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":"{\"location\":\"Paris"}
{"type":"task.tool_call_arguments.delta","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"delta":", France\"}"}
{"type":"task.tool_call_arguments.done","task_id":"call_1234xyz","item_id":"fc_1234xyz","output_index":1,"arguments":"{\"location\":\"Paris, France\"}"}
// Done
{"type":"task.output_item.done","task_id":"call_1234xyz","output_index":1,"item":{"type":"tool_call","id":"fc_1234xyz","call_id":"call_1234xyz","name":"get_weather","arguments":"{\"location\":\"Paris, France\"}"}}

// Function call output item
// Added: 初始状态下 content 和 block_list 都是空的
{"type":"task.output_item.added","task_id":"call_1234xyz","output_index":2,"item":{"type":"tool_result","id":"fco_1234xyz","call_id":"call_1234xyz","block_list":[]}}
// Deltas for block_list
// 1. 对于较短的 block，一次性传输
{"type":"task.text.done","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":0,"item":{"type":"text","text":"{\"temperature\":\"15C\",\"condition\":\"Sunny\"}","id":1}}
// 2. 对于较大的 block，带上状态进行传输
{"type":"task.image.added","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":""},"id":1}}
{"type":"task.image.done","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
// 3. 对于超大的 block，可以继续拆分 delta 进行传输
// image - added 前端开始准备传输
{"type":"task.image.added","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":""},"id":1}}
// image - deltas 传输中，每个 delta 都是一个完整的图片
{"type":"task.image.delta","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"partial_image_index":0,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
{"type":"task.image.delta","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"partial_image_index":1,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
// image - done 传输完成，最后一个图片才是最终状态的图片
{"type":"task.image.done","task_id":"call_1234xyz","item_id":"fco_1234xyz","output_index":2,"block_index":1,"item":{"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}}
// Done
{"type":"task.output_item.done","task_id":"call_1234xyz","output_index":2,"item":{"type":"tool_result","id":"fco_1234xyz","call_id":"call_1234xyz","block_list":[
  {"type":"text","text":"{\"temperature\":\"15C\",\"condition\":\"Sunny\"}","id":1},
  {"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}
]}}

// Message item
// Added
{"type":"task.output_item.added","task_id":"call_1234xyz","output_index":3,"item":{"type":"message","id":"msg_1234xyz","role":"assistant","block_list":[]}}
// Deltas for block_list
{"type":"task.text.done","task_id":"call_1234xyz","item_id":"msg_1234xyz","output_index":3,"block_index":0,"item":{"type":"text","text":"The weather in Paris is sunny with a temperature of 15C.[^1]","id":1,"annotations":[{"type":"reference_to_block","reference_id":1,"start_index":44,"end_index":47}]}}
// Done
{"type":"task.output_item.done","task_id":"call_1234xyz","output_index":3,"item":{"type":"message","id":"msg_1234xyz","role":"assistant","block_list":[{"type":"text","text":"The weather in Paris is sunny with a temperature of 15C.[^1]","id":1,"annotations":[{"type":"reference_to_block","reference_id":1,"start_index":44,"end_index":47}]}]}}
// --- 以上是子 agent 的 event stream 透传结束 ---
// 此时，前端收集到的主 agent tool_result 的 block_list 已经包含了子 agent 的所有输出部分
// {"type":"tool_result","id":"fco_1234xyz","task_id":"task_1234xyz","call_id":"call_1234xyz","block_list":[
//   {"type":"reasoning","summary":[
//     {"type":"text","text":"Thinking about the weather in Paris."},
//     {"type":"text","text":"Decided to call get_weather function."}
//   ]},
//   {"type":"tool_call","id":"fc_1234xyz","task_id":"task_1234xyz","call_id":"call_1234xyz","name":"get_weather","arguments":"{\"location\":\"Paris, France\"}"},
//   {"type":"tool_result","id":"fco_1234xyz","task_id":"task_1234xyz","call_id":"call_1234xyz","block_list":[
//     {"type":"text","text":"The current weather in Paris is sunny with a temperature of 15C.","id":1},
//     {"type":"image","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."},"id":1}
//   ]},
//   {"type":"message","id":"msg_1234xyz","task_id":"task_1234xyz","role":"assistant","block_list":[
//     {"type":"text","text":"The weather in Paris is sunny with a temperature of 15C.[^1]","id":1,"annotations":[{"type":"reference_to_block","reference_id":1,"start_index":44,"end_index":47}]}
//   ]}
// ]}
// Done: 主 agent 的 tool_result item 传输完成
{"type":"task.output_item.done","task_id":"task_1234xyz","output_index":2,"item":{"type":"tool_result","id":"fco_1234xyz","call_id":"call_1234xyz","status":"completed"}}  // <- 注意，这里只传输状态完成，不需要重复传输 block_list