Skip to content

Add moderation to OpenAI::Client

George Koltsov requested to merge georgekoltsov/add-moderation-api-openai into master

What does this MR do and why?

This MR:

  1. Adds moderations to public interface of OpenAi::Client for others to use
  2. Updates all other existing public interface methods of OpenAi::Client (chat/edits/completions/etc) (except embeddings and moderation endpoint itself) to pass any input text through OpenAI's moderations API and raise an exception if provided text violates OpenAI's Content Policy

Mentions https://gitlab.com/gitlab-org/gitlab/-/issues/408171

Testing 🔬

You can use a test string example from OpenAI documentation https://platform.openai.com/docs/api-reference/moderations/create

Moderations

Moderations 
> Gitlab::Llm::OpenAi::Client.new(User.first).moderations(input: 'kittens')

=> {"id"=>"modr-",
 "model"=>"text-moderation-004",
 "results"=>
  [{"flagged"=>false,
    ...
  }]

> Gitlab::Llm::OpenAi::Client.new(User.first).moderations(input: '...')

=> {"id"=>"modr-",
 "model"=>"text-moderation-004",
 "results"=>
  [{"flagged"=>true,
    ...
  ]}
Chat 
> Gitlab::Llm::OpenAi::Client.new(User.first).chat(content: 'good')

=> {"id"=>"chatcmpl-",
 "object"=>"chat.completion",
 "created"=>1682694183,
 ...

> Gitlab::Llm::OpenAi::Client.new(User.first).chat(content: '...')
  
Gitlab::Llm::OpenAi::Client::InputModerationError: Provided input violates OpenAI's Content Policy

> Gitlab::Llm::OpenAi::Client.new(User.first).chat(content: '...', moderated: false)

=> {"id"=>"chatcmpl-7AJwEvY9P7BC1PpWgvg1NqrH8tF3w",
 "object"=>"chat.completion",
 "created"=>1682694278,
 ...
Messages_chat 
> Gitlab::Llm::OpenAi::Client.new(User.first).messages_chat(messages: [{ role: 'user', content: 'good'}, { role: 'user', content: 'bad' }])

Gitlab::Llm::OpenAi::Client::InputModerationError: Provided input violates OpenAI's Content Policy

> Gitlab::Llm::OpenAi::Client.new(User.first).messages_chat(messages: [{ role: 'user', content: 'good'}, { role: 'user', content: 'bad' }], moderated: false)

=> {"id"=>"chatcmpl-",
 "object"=>"chat.completion",
 "created"=>1682694444,
 ...
Completions 
> Gitlab::Llm::OpenAi::Client.new(User.first).completions(prompt: 'good')

=> {"id"=>"cmpl-",
 "object"=>"text_completion",
 "created"=>1682694565,
 ...

> Gitlab::Llm::OpenAi::Client.new(User.first).completions(prompt: 'bad')

Gitlab::Llm::OpenAi::Client::InputModerationError: Provided input violates OpenAI's Content Policy

> Gitlab::Llm::OpenAi::Client.new(User.first).completions(prompt: 'bad', moderated: false)

=> {"id"=>"cmpl-",
 "object"=>"text_completion",
 "created"=>1682694606,
 ...
Edits 
> Gitlab::Llm::OpenAi::Client.new(User.first).edits(input: 'good', instruction: 'edit')

=> {"object"=>"edit", "created"=>1682694664, "choices"=>[{"text"=>"goods\n", "index"=>0}], "usage"=>{"prompt_tokens"=>14, "completion_tokens"=>14, "total_tokens"=>28}}

> Gitlab::Llm::OpenAi::Client.new(User.first).edits(input: 'bad', instruction: 'edit')

Gitlab::Llm::OpenAi::Client::InputModerationError: Provided input violates OpenAI's Content Policy

> Gitlab::Llm::OpenAi::Client.new(User.first).edits(input: 'bad', instruction: 'edit', moderated: false)

=> {"object"=>"edit", "created"=>1682694721 ...
Embeddings 
> Gitlab::Llm::OpenAi::Client.new(User.first).embeddings(input: 'good')

=> {"object"=>"list",
 "data"=>
  [{"object"=>"embedding",
...

> Gitlab::Llm::OpenAi::Client.new(User.first).embeddings(input: 'bad')

=> {"object"=>"list",
 "data"=>
  [{"object"=>"embedding",
...

> Gitlab::Llm::OpenAi::Client.new(User.first).embeddings(input: 'bad', moderated: true)

Gitlab::Llm::OpenAi::Client::InputModerationError: Provided input violates OpenAI's Content Policy

Screenshots or screen recordings

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by George Koltsov

Merge request reports