Back in January, OpenAI developed a classifier that differentiated between text authored by humans and text generated by AI systems from various providers. The project addressed concerns about using AI-generated content for misinformation campaigns, academic dishonesty, and falsely presenting AI chatbots as humans.
The classifier’s performance was not perfect, with 26% accuracy in identifying AI-written text correctly (true positives) and a 9% false positive rate of incorrectly labelling the human-written text as AI-generated. However, it improved reliability with longer input texts, especially from more recent AI systems.
OpenAI, whose ChatGPT recently debuted on Google Play Store, made this classifier publicly available to gather feedback and assess its usefulness while continuing its work on detecting AI-generated text and exploring potential improvements for the future.
The classifier had several limitations. It was unsuitable for sole decision-making, ineffective with short texts, texts in languages other than English, and code. Additionally, it could mistakenly label human-written text as AI-generated and could not reliably identify very predictable text.
Educators expressed interest in AI-written text identification. OpenAI recognized the importance of understanding the impacts and limitations of AI-generated text classifiers in educational settings. They developed a preliminary resource for educators and actively engaged with educators in the United States to gain insights and broaden their outreach to various groups, including journalists and misinformation researchers.
However, as of July 20, 2023, the AI classifier is no longer available due to its low accuracy rate. OpenAI is committed to incorporating feedback and researching more effective provenance techniques for text. They aim to develop and deploy mechanisms allowing users to determine whether audio or visual content is AI-generated.
OpenAI encourages those directly affected by these issues, such as educators, students, parents, and education service providers, to provide feedback through a form. They are particularly interested in resources developed by educators, such as course guidelines, policy updates, AI literacy programs, and other valuable insights.
In summary, OpenAI’s efforts in developing an AI-written text classifier have been met with certain limitations and challenges. While the classifier is no longer available due to its low accuracy, the research continues. OpenAI is actively seeking input from impacted communities to improve future AI systems’ reliability and address concerns surrounding AI-generated content.
