Use ChatCompletionRequest for Chat inputs

[Shiti]

Most LLM Agent libraries and services now conform to a standard interface similar to OpenAI's ChatCompletionRequest. It would make ui-components library easier to use and adopt if the input components send messages in this format instead of custom formats such as text and filesWithText.

OpenAI only documents using image and audio for content types of ChatCompletionRequest but we could make it more generic to support any file type.

For example, the interface could be something like,

interface Content {
  type: string,
  text?: string,
  url?: string,
}

export interface ChatCompletionRequest {
  messages: { content: Content[], role: string }[]
}

and some sample corresponding messages sent on the websocket could be

{
  "id": "4f6f6c72-4328-4170-8d35-7fabe322be3e",
  "timestamp": "2024-11-29T18:03:25.120Z",
  "sender": {
    "id": "46ec2629",
    "name": "Anonymous"
  },
  "conversationId": "7fabe322be3e",
  "format": "ChatCompletionRequest",
  "data": {
    "messages": [
      {
        "content": [
          {
            "type": "text",
            "text": "what was the monthly average sales reported for the last quarter?"
          },
          {
            "type": "file",
            "url": "https://somefile.xls"
          }
        ],
        "role": "user"
      }
    ]
  }
}

Pros

• Simply integration experience for developers using ui-components library with LLM Agents

Cons

• This would break existing applications

[lyjeileen]

• Currently, the message is sent without waiting for the URL to be returned. Should we change this behavior?
• Just to confirm, are the sender, id, and timestamp attributes specific to the last message in the messages.content array? Will the content continue to grow as the chat progresses?

[Shiti]

• Currently, the message is sent without waiting for the URL to be returned. Should we change this behavior?

If thats the case, yes we may need to update that

• Just to confirm, are the sender, id, and timestamp attributes specific to the last message in the messages.content array? Will the content continue to grow as the chat progresses?

The example I shared above is a single ui-components Message sent when someone clicks the send button on the text-input or multimodal-input components.

No, I dont think we should append all previous messages to the content. That is something backend should handle if desired.

[lyjeileen]

• Could you add a comparison of message types for before and after?
• I thought it was possible to have both url and text in a single type. Could you create two separate types for Content?

[Shiti]

Following are some examples of current messages sent vs future ones with this change -
Note: I have skipped the other fields in ui-components message which will not be effected.

Current Implementation	Proposed Changes
{   "format": "text",   "data": { "text": "breakdown what llms are for a non-techie"   } }	{ "format": "ChatCompletionRequest", "data": { "messages": [ { "content": [ { "type": "text", "text": "breakdown what llms are for a non-techie"    } ], "role": "user" } ] } }
{   "format": "filesWithText",   "data": { "files":[ { "name": "somefile.xls" } ]   } }	{ "format": "ChatCompletionRequest", "data": { "messages": [ { "content": [ { "type": "file", "url": "https://somefile.xls"    } ], "role": "user" } ] } }
{   "format": "filesWithText",   "data": { "text": "what was the monthly average sales reported for the last quarter?" "files":[ { "name": "somefile.xls" } ]   } }	{ "format": "ChatCompletionRequest", "data": { "messages": [ { "content": [ { "type": "text", "text": "what was the monthly average sales reported for the last quarter?"    } { "type": "file", "url": "https://somefile.xls"    } ], "role": "user" } ] } }

I thought it was possible to have both url and text in a single type. Could you create two separate types for Content?

@lyjeileen yes, I agree, it would be simpler to define two types for the content with clear indication of the fields.

[lyjeileen]

This makes sense. Does this also mean we are going to replace Text and Multipart components with ChatCompletionRequest component?

[Shiti]

I was thinking of updating Multipart and leaving Text as is. Or I could avoid changing those but would need to have something else for displaying ChatCompletionRequest . What do you think ?

[lyjeileen]

Do you think Multipart would still be useful to others? If the backend normally sends messages in the ChatCompletionRequest format, it makes sense to update Multipart.

[Shiti]

We decided to keep the Multipart and introduced a separate component for ChatCompletion. The changes discussed were released in v.0.0.34