Rewrite the AI feature development playbook

What does this MR do and why?

It moves the content of the existing AI feature development playbook to a new file named prompt_engineering.md and replaces the content of the playbook with an overview of AI feature development phases/steps.

This MR is intended to help us:

• improve our guidelines for engineers for evaluating AI features
• create an SSoT for the AI feature development workflow that's consistent with the overall product development workflow
• provide an overview, with links to more detailed information
• take step towards consolidating AI dev docs (#514510 (closed))

It's not meant to be merged as is (it's an early draft), but to give us a place to discuss the content and add/edit as needed until it can be merged.

Background

The existing AI feature development playbook provides some information about some aspects of AI feature development, including evaluation, but it isn't quite the SSoT for AI feature development that it could be.

At minimum, teams need to know:

1. A high-level summary of what's involved (evaluation, but also everything else including planning, development (including prompt eng), testing, deployment/monitoring, etc)
2. Summaries of each phase, with links to detailed guides (some existing, some need to be revise, some will need to be written), so they can find the information they need without having to look in many different places
3. Links to reference materials (external and internal) to enable a deeper understanding

Proposal for this MR

• Review how the new version of the playbook is organized, and decide if it is suitable, or if there's a better way to organize/present the information.

TODO:

• replace the content of ai_feature_development_playbook.md with the updated content from this MR
• submit it for documentation review after review by domain experts

Follow-up:

• Adding documentation for Key Activities that are just bullet points in this MR
• Updating the Evaluation documentation - that's next in gitlab-org/modelops/ai-model-validation-and-research/ai-evaluation/prompt-library#674 (closed)
• Move docs from the internal handbook to the public documentation

References

• Update the AI feature development playbook (gitlab-org/modelops/ai-model-validation-and-research/ai-evaluation/prompt-library#743 - closed)
• Document developer workflow to enable efficient... (gitlab-org/modelops/ai-model-validation-and-research/ai-evaluation/prompt-library#674 - closed)
• Nav update: Add Prompt engineering to the global nav (gitlab-org/technical-writing/docs-gitlab-com!810 - merged)

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.