ChatML — New Prompt Format for the ChatGPT API

OpenAI’s recent March 1, 2023 announcement introduced the ChatGPT and Whisper APIs.
The update also included a 10X price drop for hosted GPT-3 service via the ChatGPT API.
Also introduced during the announcement was ChatML, the format required for inputs to the ChatGPT API.

Importance of ChatML Language

ChatML addresses LLMs’ main security vulnerability and avenue of abuse: prompt injection attacks.
ChatML segregates conversation into different layers or roles (system, assistant, user)
OpenAI plans to develop ChatML further, accommodating other data types
Using the unsafe raw string format still allows injections, but ChatML makes explicit the source of each piece of text, mitigating the problem.
ChatML’s explicit identification of the origin of each piece of text improves the ease of sequencing prompts into chains for conversational interfaces.
There is also a plaintext version of ChatML, but it is much more susceptible to prompt injection.

Roles in ChatML

Roles improve ChatGPT’s capability to differentiate between safe/trusted inputs and those that may contain prompt injection attacks.
There are three primary roles, system, assistant, and user.

The `system` role

The system role also accepts an additional name property that can be used for few-shot (k-shot) prompting. For example, you may use the names example_user and example_assistant.
Note the current model, gpt-3.5-turbo-0301, doesn’t pay “strong” attention to the system role.
- Sometimes, it may be best to use the user role instead.
- However, the system role is currently being improved and should be more useful soon.

Example ChatML

The following is an example of ChatML in the JSON format:

[
	{
		"role": "system",
		"content": "You are the helpful curator of a virtual art museum."
	},
	{
		"role": "user",
		"content": "Can you show me the famous paintings here?"
	},
	{
		"role": "assistant",
		"content": "Sure, we have 'Starry Night' by Vincent van Gogh, 'Mona Lisa' by Leonardo da Vinci and 'The Persistence of Memory' by Salvador Dali."
	},
	{
		"role": "user",
		"content": "Can you tell me more about 'Starry Night'?"
	},
	{
		"role": "assistant",
		"content": "'Starry Night' is an oil painting by Vincent van Gogh which depicts the view from his asylum room in Saint-Paul-de-Mausole in southern France at night. It's one of his most famous works and is known for its swirling, dreamlike composition and vivid colors."
	},
	{
		"role": "user",
		"content": "Thank you, can you also show me the sculpture section?"
	}
]

This example demonstrates the few-shot learning prompt using names in the system role:


[
	{
		"role": "system", 
		"content": "You are a knowledgeable and reliable AI assistant that can help you manage your schedule and keep track of your tasks."
	},
	{
		"role": "system", 
		"name":"example_user", 
		"content": "Can you please schedule a meeting with the marketing team for next Monday at 10am?"
	},
	{
		"role": "system", 
		"name": "example_assistant", 
		"content": "Sure, I've scheduled a meeting with the marketing team for next Monday at 10am. Anything else I can help you with?"
	},
	{
		"role": "system", 
		"name":"example_user", 
		"content": "Yes, can you also remind me to send the weekly report to the CEO by Friday?"
	},
	{
		"role": "system", 
		"name": "example_assistant", 
		"content": "Of course, I'll send you a reminder on Thursday to send the weekly report to the CEO by Friday. Is there anything else you need help with?"
	},
	{
		"role": "user", 
		"content": "Can you tell me what meetings I have this week?"
	},
]