Update AI local step up docs
Problem to solve
We forgot to adjust the following verification step for testing AI features locally in the GDK. We need to modify the prompt message to reflect the correct AI-Gateway format:
Gitlab::Llm::AiGateway::Client.new(User.first).stream(prompt: [{role: "user", content: "Hi, how are you?"}])
Issues
Looks like there is some requirement to having a Cloud Access token set in ruby console. The issue is getting this token saved doesn't correct work.
User Load (1.5ms) SELECT "users".* FROM "users" ORDER BY "users"."id" ASC LIMIT 1 /*application:console,db_config_name:main,console_hostname:Nathans-MacBook-Pro-2.local,console_username:nweinshenker,line:/app/models/concerns/use_sql_function_for_primary_key_lookups.rb:8:in `_query_by_sql'*/
CloudConnector::ServiceAccessToken Load (0.2ms) SELECT "service_access_tokens".* FROM "service_access_tokens" WHERE (expires_at > '2024-05-15 18:39:33.341293') ORDER BY "service_access_tokens"."id" DESC LIMIT 1 /*application:console,db_config_name:main,console_hostname:Nathans-MacBook-Pro-2.local,console_username:nweinshenker,line:/ee/app/services/cloud_connector/access_service.rb:40:in `self_managed_token'*/
=> nil
Reference Links
Edited by Nathan Weinshenker