How I work: Cloze substitution

There have been a number of high-profile cases of employees sharing sensitive company information with LLMs like ChatGPT. And while the material Orchestra generates is intended for public use and isn't likely to involve sensitive or trade secrets, data you enter can be used to train OpenAI's models (although recently the company announced a feature to let you turn this off, it's still on by default).

The best way to ensure a system can't use data it shouldn't have access to is to not give it access in the first place, so at Orchestra we're incorporating a technique called Cloze substitution in our platform to ensure that identifying information can't be used in the platform. Company, product, and staff names, among a few other items, are all associated with a unique identifier in the Orchestra platform.

When we generate material for you we incorporate the information you enter with several prompts we have built, but as a final measure we substitute specific names with generic but unique strings. Then, in the output, we reverse the process. When the LLM generates a response it will return these strings. We take the response and reverse the substitution so that the material looks correct to end users. I asked ChatGPT to write up a little javascript pseudocode and an explanation to help illustrate the concept:

const sensitiveText = "Our company, Acme Inc., is launching a new product called Xylo. Our employee, John Smith, has been leading the development team.";
const sensitiveWords = { "Acme Inc.": "COMPANY", "Xylo": "PRODUCT", "John Smith": "EMPLOYEE"};

function anonymizeText(text, words) 
  { for (let key in words) { 
      const regex = new RegExp(key, "gi"); 
      text = text.replace(regex, words[key]); 
   } 
   return text;
}

/*A made-up method call to an LLM  */
function generateResponse(text) { 
  const response = LLM.generateText(text); 
  return response;
}

function restoreText(text, words) { 
   for (let key in words) { 
      const regex = new RegExp(words[key], "gi"); 
      text = text.replace(regex, key); 
   } 
   return text;
}
const anonymizedText = anonymizeText(sensitiveText, sensitiveWords);
const response = generateResponse(anonymizedText);
const restoredText = restoreText(response, sensitiveWords);

console.log(restoredText);

In this example, we have a sensitive text that includes words related to a company name, product name, and an employee name. We define an object of sensitive words that we want to replace with unique identifiers. We then define a function that takes in the sensitive text and the sensitive words object as arguments. The function uses a regular expression to replace each occurrence of the sensitive words with the corresponding unique identifier. Finally, the function returns the anonymized text.

Next, we define a function generateResponse which sends the anonymized text to an LLM using a made-up method call and returns the response.

Finally, we define a function restoreText which takes in the response and the sensitive words object as arguments. The function uses a regular expression to replace each occurrence of the unique identifier with the corresponding sensitive word. Finally, the function returns the restored text.

We then call the anonymizeText function to anonymize the sensitive text, generate a response from the anonymized text using the generateResponse function, and then restore the original text using the restoreText function. The restored text includes the substituted words, making a whole sentence from the response.

Note that the LLM method generateText is made up for the purposes of this pseudocode example, and would need to be replaced with a real method call for an actual implementation.

While the risks may be small given the material customers provide to Orchestra is designed to be shared broadly, it's still never a bad practice to build security in. After all, there are some capabilities in these tools that aren't even fully understood by their creators. Cow.