Skip to content

Draft: docs(AIHumanInteraction): add guidance informing user how system works

Katie Macoy requested to merge katiemacoy-main-patch-47351 into main

Context

This MR comes from two pieces of feedback from @tichen

Feedback 1/2

The guidelines currently state Tell the user how their data is used and processed.

Tiffany's feedback:

Do you think we should remove this line? For context, these are Design's general guidelines for AI features that require AI-human interaction.

Defer to you on the best approach here. It might be a good idea to remove this line so that we centralize our statements re data processing and prevent situations where each feature is describing its own data processing practices in different ways. From my understanding, most AI features will probably process personal data in similar ways that can be addressed in a centralized way.

Feedback 2/2

The guidelines currently state Explain what data the system is trained on and what it's optimized for.

Tiffany's feedback:

can we remove this particular line? This won't always be possible as some third party AI services that we use aren't 100% transparent as to what their models were trained on. We also might not always want to do this even if it's possible.

What this MR aims to accomplish

I understand Tiffany's feedback from a legal perspective. I would like to strike a balance between legal and user experience. The original intention of the two guidelines above was to be very transparent about how AI systems work so users can understand them to build a mental model and also make informed decisions like:

  • how much can I trust this system?
  • how did this system generate this recommendation?
  • should I use this system knowing it's going to send my data to a third-party?

It will rarely be feasible or practical to explain in depth how and AI system works or how user data is processed. However, we want to proactively surface some information in the UI so users can build trust in the system and make informed decisions. I've re-written the guidelines to (hopefully) focus more on being transparent where possible about how a system works so the user can make informed decisions. I've also tried to make it more clear we do not need to go into the fine-grained details about how a system works.

I'm not sure I've completely nailed this message so any feedback is welcome. This MR is in draft.

Merge request reports