🌴 Brian

ChatML — New Prompt Format for the ChatGPT API

Mar 01, 20233 min read

Featured Image

  • OpenAI’s recent March 1, 2023 announcement introduced the ChatGPT and Whisper APIs.

  • The update also included a 10X price drop for hosted GPT-3 service via the ChatGPT API.

  • Also introduced during the announcement was ChatML, the format required for inputs to the ChatGPT API.

Importance of ChatML Language

  • ChatML addresses LLMs’ main security vulnerability and avenue of abuse: prompt injection attacks.

  • ChatML segregates conversation into different layers or roles (system, assistant, user)

  • OpenAI plans to develop ChatML further, accommodating other data types

  • Using the unsafe raw string format still allows injections, but ChatML makes explicit the source of each piece of text, mitigating the problem.

  • ChatML’s explicit identification of the origin of each piece of text improves the ease of sequencing prompts into chains for conversational interfaces.

  • There is also a plaintext version of ChatML, but it is much more susceptible to prompt injection.

Roles in ChatML

  • Roles improve ChatGPT’s capability to differentiate between safe/trusted inputs and those that may contain prompt injection attacks.

  • There are three primary roles, system, assistant, and user.

The system role

  • The system role also accepts an additional name property that can be used for few-shot (k-shot) prompting. For example, you may use the names example_user and example_assistant.

  • Note the current model, gpt-3.5-turbo-0301, doesn’t pay “strong” attention to the system role.

    • Sometimes, it may be best to use the user role instead.

    • However, the system role is currently being improved and should be more useful soon.

Example ChatML

The following is an example of ChatML in the JSON format:

[
	{
		"role": "system",
		"content": "You are the helpful curator of a virtual art museum."
	},
	{
		"role": "user",
		"content": "Can you show me the famous paintings here?"
	},
	{
		"role": "assistant",
		"content": "Sure, we have 'Starry Night' by Vincent van Gogh, 'Mona Lisa' by Leonardo da Vinci and 'The Persistence of Memory' by Salvador Dali."
	},
	{
		"role": "user",
		"content": "Can you tell me more about 'Starry Night'?"
	},
	{
		"role": "assistant",
		"content": "'Starry Night' is an oil painting by Vincent van Gogh which depicts the view from his asylum room in Saint-Paul-de-Mausole in southern France at night. It's one of his most famous works and is known for its swirling, dreamlike composition and vivid colors."
	},
	{
		"role": "user",
		"content": "Thank you, can you also show me the sculpture section?"
	}
]

This example demonstrates the few-shot learning prompt using names in the system role:


[
	{
		"role": "system", 
		"content": "You are a knowledgeable and reliable AI assistant that can help you manage your schedule and keep track of your tasks."
	},
	{
		"role": "system", 
		"name":"example_user", 
		"content": "Can you please schedule a meeting with the marketing team for next Monday at 10am?"
	},
	{
		"role": "system", 
		"name": "example_assistant", 
		"content": "Sure, I've scheduled a meeting with the marketing team for next Monday at 10am. Anything else I can help you with?"
	},
	{
		"role": "system", 
		"name":"example_user", 
		"content": "Yes, can you also remind me to send the weekly report to the CEO by Friday?"
	},
	{
		"role": "system", 
		"name": "example_assistant", 
		"content": "Of course, I'll send you a reminder on Thursday to send the weekly report to the CEO by Friday. Is there anything else you need help with?"
	},
	{
		"role": "user", 
		"content": "Can you tell me what meetings I have this week?"
	},
]

References

Graph View

Backlinks

  • No backlinks found