Amazon warns employees not to share confidential information with ChatGPT after seeing cases where its answer 'closely matches existing material' from inside the company

January 26, 2023

Last month, an internal Slack channel at Amazonbustled with employee questions about ChatGPT, the artificial intelligence tool that's taken the tech world by storm since its late-November release.

Some asked whether Amazon had official guidance on using ChatGPT on work devices. Others wondered if they were even allowed to use the AI tool for work. One person urged the Amazon Web Services cloud unit to publish its position on "acceptable usage of generative AI tools," like ChatGPT.

Soon, an Amazon corporate lawyer chimed in. She warned employees not to provide ChatGPT with "any Amazon confidential information (including Amazon code you are working on)," according to a screenshot of the message seen by Insider.

The attorney, a senior corporate counsel at Amazon, suggested employees follow the company's existing conflict of interest and confidentiality policies because there have been "instances" of ChatGPT responses looking similar to internal Amazon data.

"This is important because your inputs may be used as training data for a further iteration of ChatGPT, and we wouldn't want its output to include or resemble our confidential information (and I've already seen instances where its output closely matches existing material)," the lawyer wrote.

The exchange reflects one of the many new ethical issues arising as a result of the sudden emergence of ChatGPT, the conversational AI tool that can respond to prompts with markedly articulate and intelligent answers. Its rapid proliferation has the potential to upend a number of industries, across media, academics, and healthcare, precipitating a frenzied effort to grapple with the chatbot's use-cases and the consequences.

The question of how confidential company information is shared with ChatGPT and what OpenAI, the creator of the AI tool, does with it could become a thorny issue going forward. It's particularly important for Amazon as its main competitor Microsoft has invested heavily in OpenAI, including a fresh round of funding this week that reportedly totals $10 billion.

"OpenAI is far from transparent about how they use the data, but if it's being folded into training data, I would expect corporations to wonder: After a few months of widespread use of ChatGPT, will it become possible to extract private corporate information with cleverly crafted prompts?" said Emily Bender, who teaches computational linguistics at University of Washington.

Amazon's spokesperson didn't respond to a request for comment. OpenAI's representative pointed to ChatGPT's FAQ page for any questions regarding its data and privacy policies.

Some Amazon staff already use ChatGPT as a 'coding assistant'

Amazon has put in some internal guardrails for ChatGPT. For example, when employees use a work device to go to the ChatGPT website, a warning message pops up, saying they're about to visit a third-party service that "may not be approved for use by Amazon Security," according to screenshots seen by Insider.

Employees in the Slack channel said they were able to bypass that message by simply clicking on the "Acknowledge" tab. Staff surmised the warning pop-up is there to prevent employees from pasting confidential information to ChatGPT, especially since they've seen no company policy yet on its internal use.

Still, some Amazonians are already using the AI tool as a software "coding assistant" by asking it to improve internal lines of code, according to Slack messages seen by Insider.

"I think it would be awesome to be able to use this directly now," one person wrote in the Slack channel. "So any guidance would be fantastic."

'Both scared and excited'

Another employee said he shared with ChatGPT some of Amazon's interview questions for an open coding position. The model gave the correct solution to several of these technical questions, according to this employee's posts on Slack.

"I'm both scared and excited to see what impact this will have on the way that we conduct coding interviews," this staffer wrote on Slack.

Overall, Amazon employees in the Slack channel were excited about the potential of ChatGPT, and wondered if Amazon was working on a competing product.

The corporate lawyer who warned employees about using ChatGPT said Amazon was broadly developing "similar technology," citing the voice-assistant Alexa and the code recommendation service CodeWhisperer.

One AWS employee wrote that the Enterprise Support team recently started a small working group internally to "understand the impact of advanced chat AI on our business," according to the Slack messages. The study revealed that ChatGPT "does a very good job" at answering AWS support questions, including difficult ones like troubleshooting Aurora database problems. It's also "great" at creating training material for AWS Certified Cloud Architect exams and "very good" at coming up with a customer's company goals, the employee Slack messages stated.

'Far from transparent'

The increased use of ChatGPT at work raises serious questions about how OpenAI plans to use the material shared with the AI tool, according to Bender from the University of Washington.

OpenAI's terms of service require users to agree that the company can use all input and output generated by the users and ChatGPT. It also says it removes all personally identifiable information (PII) from the data it uses.

Bender said it's hard to see how OpenAI is "thoroughly" identifying and removing personal information, given ChatGPT's rapidly growing scale -- it crossed 1 million users within a week of launching. More importantly, intellectual property of corporations is likely not part of what is defined under PII, Bender said.

For Amazon employees, data privacy seems to be the least of their concerns. They said using the chatbot at work has led to "10x in productivity," and many expressed a desire to join internal teams developing similar services.

"If there is a current initiative to build a similar service I would be interested in committing time to helping build it if needed," one of the employees wrote in Slack.