Updates Agent prompt to match Anthropic API

What does this MR do and why?

AI Agents currently use the default model (which is Claude), but the generated system prompts were not matching the API https://docs.anthropic.com/claude/reference/prompt-validation. Additionally, we do some post processing after a message was a received, but the instructions were not present in the generated prompt. This comes at the cost that the user does not have full control of what is being sent to the LLM, which is ok as a temporary limitation until we have more models to choose from.

• Before:

\n\nHuman: Question 1\n\nAssistant: Answer 1\n\nHuman: I am a pirate and always speak like a pirate, and I have a parrot named David.\n\nQuestion: test\nThought: \n

• After:

\n\nHuman: Question 1\n\nAssistant: Answer 1\n\nHuman: I am a pirate and always speak like a pirate, and I have a parrot named David.\n\nYou must always use the following format:\nQuestion: the input question you must answer\nThought: you should always think about what to do\nAction: the action to take, should be one tool from this list or a direct answer (then use DirectAnswer as action): [IssueReader, GitlabDocumentation, EpicReader, CiEditorAssistant]\nAction Input: the input to the action needs to be provided for every action that uses a tool\nObservation: the result of the actions. If the Action is DirectAnswer never write an Observation, but remember that you're still GitLab Duo Chat.\n\n... (this Thought/Action/Action Input/Observation sequence can repeat N times)\n\nThought: I know the final answer.\nFinal Answer: the final answer to the original input question.\n\nWhen concluding your response, provide the final answer as "Final Answer:" as soon as the answer is recognized.\n\nBegin!\n\nQuestion: test\nThought: \n

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

How to set up and validate locally

1. Ensure you have an EE license activated

2. Enable the feature flag and populate the database:

   Feature.enable(:agent_registry)
   p = Project.find_by_id(1)
   agent = Ai::Agent.create!(project: p, name: 'agent_1')
   Ai::AgentVersion.create!(agent: agent, project_id: 1, model: 'default', prompt: 'Awesome prompt')

3. Open the logs tail -f log/llm.log | jq. You can now see the prompts that are being called

4. Load /-/ml/agents for the project used to create the agent, and click on the agent agent_1

5. On the chat box, type anything (eg test) and verify on the logs that the prompt created is correct

Related to #441658 (closed)