Agentic Chat should not repeat the same large context for every message

Problem

When agentic chat is used in a the Web UI we often extract a bunch of context from the thing the user is looking at and send that along to the LLM. For example if they are viewing an issue we extract the issue title, description and all comments. For large issues this is a lot of context tokens to use up. This is intentional and there are questions about whether or not we should just be including a link so the LLM can look it up. There are cases where the user may not even be asking about the issue and then it's wasted context.

But leaving that tradeoff aside we observe there is a bigger issue that we keep duplicating that context every time a message is sent in the same thread. So the 2nd message in a thread includes all the issue details twice. And the 3rd three times (and so-on). As such the context grows way too quickly when looking at large issues.

See Slack thread (internal) for more discussion.

How to reproduce

1. Start a new agentic chat thread on an issue or MR
2. Say hello
3. Say hello
4. Say hello
5. Check the langsmith logs and you will see the <additional_context> stuff included in the last 3 human messages
6. You will also see that the number of tokens growing quite rapidly

Solution

At a minimum we should be de-duplicating this content. We have the following options:

1. In the Duo Workflow Service we explicitly check for <additional_context> tags and remove those that are duplicated. We should only keep the latest version of it.
2. In the Duo Workflow Service always remove <additional_context> tags from human messages that aren't the latest one.
3. In the client (before sending the <additional_context> to the Duo Workflow Service we detect if the <additional_context> is the same as what the user was looking at when they sent the last message. If so we do not include it with the goal

All options are a little hacky because we'll be spreading the understanding of the meanting of <additional_context> across different places.

Option (1) is likely the best option.

Option (3) is likely not very robust because the client doesn't really have all the history of what the user has been looking at in the past.

I also noticed in the langsmith tracing (as you can see in the picture above) that each <additional_context> has a special <id> </id> tag in it. It seems this is empty. If this was populated we could be de-duplicating on this id tag. Perhaps that's what it was intended for to begin with? I'm not sure where that's being dropped but there seems to be other issues with these tags. Like everything is tagged as repository with <category> repository </category> even though I'm looking at an issue.