Support AI completions via OpenRouter API
Feature Request
Summary
OpenRouter is a service that provides a very similar API to OpenAI for completions, but supports a lot of LLM models from different providers all under the same API and with shared credits. It's been very useful for me in other applications like Cursor as well because you can just swap out the API endpoint URL and get access to tons of models.
It looks like iTerm relies on some specifics of the OpenAI API response that OpenRouter does not have - but it should be a small-ish change to support it. Are you open to supporting OpenRouter?
Alternatives
Perhaps there are other providers we could consider or more general API specs?
Details
So I tried setting the URL for AI API to "https://openrouter.ai/api/v1/completions", setting the model name to something like "anthropic/claude-3-haiku" and then trying a codecierge command. It fails with this error:
Assistant outputs: "There was an error: The operation couldn’t be completed. (iTerm2SharedARC.AIConversation.AIError error 1.)"
So looking at the OpenAI API vs. the OpenRouter API, it looks like the code expects an "index" field within the "completions" objects from the response that OpenRouter does not provide. There may be other minor differences in API that iTerm relies on. Excerpts from debug log below:
1716268831.425036 AITerm.swift:313 (handle(event:legacy:)): handle(error(Failed to decode API response: keyNotFound(CodingKeys(stringValue: "index", intValue: nil), Swift.DecodingError.Context(codingPath: [CodingKeys(stringValue: "choices", intValue: nil), _JSONKey(stringValue: "Index 0", intValue: 0)], debugDescription: "No value associated with key CodingKeys(stringValue: "index", intValue: nil) ("index").", underlyingError: nil)). Data is:
{"id":"gen-xS7jzUXU6j95Og532szrhczqUfpM","model":"anthropic/claude-3-haiku","object":"chat.completion","created":1716268828,"choices":[{"text":"Okay, let's start with a simple command to find all Python files in the current directory:\n\n\nfind . -name \"*.py\"\n\n\nThis command will search the current directory (represented by the .) and all its subdirectories for files with the .py extension.\n\nThe find command is used to search for files and directories that match certain criteria. The -name \"*.py\" part tells find to look for files with the .py extension.\n\nGive this a try and let me know if you need any further assistance.","finish_reason":"end_turn"}],"usage":{"prompt_tokens":162,"completion_tokens":113,"total_tokens":275}})) in state querySent
I might be able to send in a PR if this is something you are interested in. Thanks!