Skip to content

fix(memory): store completed BidiAgent transcripts - #722

Open
abhiramshaji-cyber wants to merge 4 commits into
aws:mainfrom
abhiramshaji-cyber:fix/memory-bidi-transcripts
Open

abhiramshaji-cyber wants to merge 4 commits into
aws:mainfrom
abhiramshaji-cyber:fix/memory-bidi-transcripts

Conversation

@abhiramshaji-cyber

@abhiramshaji-cyber abhiramshaji-cyber commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Voice agents now save what the caller and the assistant said into AgentCore Memory, in the order it was said, so long term memory has something to learn from.

What this code is for

The AgentCore Memory session manager listens to Strands agent events and writes each conversation turn to AgentCore Memory as a USER or ASSISTANT event. Memory strategies (summary, preferences, facts) read those events to build long term memory, and a restored session reads them back to rebuild the conversation.

Before this PR

  1. A caller phones a hotel voice agent built with BidiAgent and says "Any rooms in London?"
  2. BidiAgent adds an empty message for the caller's words to its history and fires MessageAddedEvent with bidi status "pending".
  3. The session manager tries to save that empty message. It has no text, so nothing is written.
  4. The caller's words are fully written down. BidiAgent fills in the message and fires MessageUpdatedEvent with status "complete".
  5. The session manager does not listen to MessageUpdatedEvent, so the caller's words are never saved. The assistant's reply "Yes, two rooms." goes through the same steps and is not saved either.
  6. Memory has no record of the call, and long term memory has nothing to learn from.

After this PR

  1. A caller phones a hotel voice agent built with BidiAgent and says "Any rooms in London?"
  2. BidiAgent adds an empty message for the caller's words to its history and fires MessageAddedEvent with bidi status "pending".
  3. The session manager does not save that message yet. Messages added after it, including the assistant's reply if that finishes first, are saved only after it is saved.
  4. The caller's words are fully written down. BidiAgent fills in the message and fires MessageUpdatedEvent with status "complete".
  5. The session manager saves the caller's words, then every finished message added after them, in the order they appear in the agent's history.
  6. Memory holds the question and then the answer, and a restored session replays them in that order.

The diagram

flowchart LR
  subgraph BEFORE
    direction TB
    B1["Caller asks: Any rooms in London?"] --> B2["Empty message added, MessageAddedEvent"]
    B2 ==> B3["Empty message not saved"]
    B3 --> B4["Words written down, MessageUpdatedEvent"]
    B4 --> B5["Update ignored, words never saved"]
    B5 --> B6["Memory has no record of the call"]
  end
  subgraph AFTER
    direction TB
    A1["Caller asks: Any rooms in London?"] --> A2["Empty message added, MessageAddedEvent"]
    A2 ==> A3["THE CHANGE: not saved yet, later messages saved only after it"]
    A3 --> A4["Words written down, MessageUpdatedEvent"]
    A4 --> A5["Saved, then later finished messages saved in history order"]
    A5 --> A6["Memory holds question, then answer"]
    N1["Finished text or tool message added, nothing unsaved before it"] --> N2["Saved right away on MessageAddedEvent, as before"]
  end
  B1 ~~~ A1
  classDef change fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px
  class A3 change
  linkStyle 1 stroke:#c62828,stroke-width:3px
  linkStyle 6 stroke:#2e7d32,stroke-width:3px
Loading

The change

In session_manager.py:

  1. append_message keeps a message whose bidi status is "pending" in a short waiting list instead of writing it. While anything is in that list, later messages are added to the end of it. When the list is empty, messages are written straight away exactly as before.
  2. A new _complete_message callback handles MessageUpdatedEvent. A "complete" update fills in the waiting message, and every ready message at the front of the list is written in order. An "incomplete" update (Strands sends one for each transcript still open when the model stream ends) removes its message without saving it, so the messages after it can be written. Any other update, including one without a bidi status, leaves the list unchanged.
  3. BidiAgentStopEvent and close() write every finished message still in the list and drop messages that never finished, so nothing finished is lost when a session ends with a turn still open.
  4. A failed write is raised as before and that one message is dropped, so the messages after it are not blocked. They are written on the next message, when the agent stops, or on close().
  5. The callbacks are registered in sync mode and, through the existing asyncio.to_thread wrapper, in async mode. A lock keeps the order when async mode runs the callbacks on worker threads.

Version bump: MessageUpdatedEvent first shipped in strands-agents 1.57.1. The minimum is raised from 1.56.0 to 1.57.1 in pyproject.toml, and uv.lock is relocked to exactly 1.57.1, so CI now runs every new test instead of skipping them. MessageUpdatedEvent is now imported directly. Version 1.57.1 also added _held_records to the Strands base session manager and reads agent.messages when syncing the agent. The test helper skips the base class constructor and some tests use bare mock agents, so the helper now sets _held_records and those agents get an empty message list. Production code calls the base constructor and is not affected.

A question for you: on newer strands-agents versions (I checked 1.62.0), BidiAgent also streams finished text replies and reasoning text through the same pending and complete messages, so with this PR those are saved as ASSISTANT events too, reasoning included. I was not sure whether you want reasoning stored in memory. Happy to filter reasoning out if you'd rather not store it.

What changes for a user

Voice agents that use BidiAgent with AgentCore Memory now keep a record of each conversation in the order it happened, and long term memory strategies can summarize it and remember the caller in later sessions. This also holds when the caller's words are written down after the assistant has started or finished replying.

What still works: regular agents and finished text or tool messages are saved the moment they are added, as before. Batching, async mode and restoring a session are unchanged.

One thing to know: while the caller's words are still being written down, the messages added after them are kept only in the running process. They are written when the words finish, when the agent stops, or on close(). If the process crashes before then, those messages are lost along with the unfinished words.

Projects pinned below strands-agents 1.57.1 need to upgrade.

Not in this PR, as the issue also mentions: the first audio of a reply still waits for one memory write in async mode, and a failed memory write can still end the voice session. Those can be handled separately.

Closes #716

BidiAgent adds each spoken turn to history as an empty placeholder
(MessageAddedEvent, bidi status "pending") and later replaces it with
the finished text through MessageUpdatedEvent (status "complete").
The session manager only persisted on MessageAddedEvent, so the
placeholder was dropped as empty and the finished text was never
written. Voice agents therefore stored no user or assistant turns and
long term memory strategies had nothing to extract.

Persist MessageUpdatedEvent messages whose bidi status is "complete"
through append_message, the same path text messages use, in both sync
and async mode. Pending placeholders are skipped explicitly so a custom
converter cannot store them, and "incomplete" updates are ignored.
Each transcript completes once, so it is written exactly once. The hook
is only registered when the installed strands-agents provides
MessageUpdatedEvent.

Closes aws#716
@abhiramshaji-cyber

Copy link
Copy Markdown
Contributor Author

@jariy17 when you get a chance, could you approve the waiting integration tests here? I reran the reproduction from #716 against Strands 1.62.0, and memory now keeps the user and assistant turns, each once.

MessageUpdatedEvent, which BidiAgent uses to report a finished
transcript, first shipped in strands-agents 1.57.1. The lock pinned
1.56.0, so CI skipped every test that exercises the transcript fix.
Raise the minimum to 1.57.1 and relock to exactly that version.

strands-agents 1.57.1 also adds RepositorySessionManager._held_records
and walks agent.messages in sync_agent. The test helper patches the
parent __init__ away and some tests use bare Mock agents, so set
_held_records in the helper and give those agents an empty message
list. Production code calls the parent __init__ and is unaffected.
BidiAgent reserves a message's place in history when a transcript
starts and fills it in when the transcript finishes. With Gemini and
OpenAI realtime the caller's transcript can still be open while the
assistant reply starts and finishes, so writing each transcript when it
finished stored the reply before the question, and a restored session
replayed them in that order.

Hold every message that comes after an unfinished BidiAgent message and
write them in history order once it finishes. An incomplete transcript
releases the messages behind it without being stored, and
BidiAgentStopEvent and close() write anything still held so nothing is
lost when a session ends with a transcript open. A failed write is
dropped and raised as before, so it cannot block later messages.
Regular Agent messages never wait because nothing is pending.

Import MessageUpdatedEvent directly now that strands-agents 1.57.1 is
the minimum, give the update callback its real event type, and add
tests that fail without the fix, including one with a converter that
keeps empty content to show why pending placeholders are never written.
A MessageUpdatedEvent that matched a waiting BidiAgent transcript but
carried no bidi status fell into the delete branch, so the transcript was
dropped and the messages behind it were written ahead of it. Only an
"incomplete" status means the transcript will never finish; any other
update now leaves it waiting until it completes or the agent stops.

Also cover history order when batch_size is above one, in sync and
async mode, including messages released when the agent stops.
@abhiramshaji-cyber

Copy link
Copy Markdown
Contributor Author

I pushed an update. Turns are now saved in history order, so a reply that finishes before the caller's words no longer lands first. I also raised the strands-agents minimum to 1.57.1 so CI actually runs the new tests. Happy to drop the version bump if you would rather handle it separately.

@github-actions github-actions Bot added size/m PR size: M and removed size/m PR size: M labels Oct 29, 2026

This branch is waiting to be deployed

1 waiting deployment
manual-approval — c386c2ff Waiting Oct 29, 2026 by abhiramshaji-cyber via Test (policy) #1677
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m PR size: M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AgentCoreMemorySessionManager: BidiAgent transcripts are never stored (MessageUpdatedEvent not handled)

1 participant