Harnessing generative AI for overcoming labeled data challenges in social media NLP

Abstract

With the introduction of Transformers and Large Language Models, the field of NLP has significantly evolved. Generative AI, a prominent transformer-based technology for crafting human-like content, has proven powerful skills across numerous NLP tasks. Simultaneously, social media emerges as a rich source for NLP explorations, offering vast and diverse datasets that capture real-time language usage, making it a valuable resource for understanding and advancing NLP techniques. Given that supervised learning is the most popular Machine Learning training method, numerous NLP studies necessitate labor-intensive annotation of social media text. However, despite the large amount of data available, the social media data annotation process is usually difficult for human experts due to unique characteristics of text, such as shortness, lack of context, embedded socio-cultural perspectives, and varied writing styles. The challenges in constructing labeled social media datasets often result in a scarcity of labeled data and the generation of low-quality labels. Moreover, these datasets frequently face class imbalance due to the limitations of labeled samples. Hence, ensuring a balanced, high-quality dataset in sufficient quantities is crucial for the robust and accurate development of NLP models. To address these challenges, this study has identified the usage of generative AI for social media labeled text generation. Specifically, this study focuses on two key objectives: augmenting existing labeled text samples and annotating unlabeled text samples using generative AI. As the generative AI technology, the Generative Pre-trained Transformer model, a prevalent choice for AI-based content generation is employed in different versions throughout the study and evaluated its performance against traditional text augmentation and annotation methods. While both studies centered around multi-class classification problems, the text augmentation approach delves into augmenting human wellness dimensions using Reddit posts, and text annotation tackles stance detection on abortion legalization using Twitter posts. By employing various classifiers, the subsequent investigations aim to enhance classification performance in social media NLP, emphasizing the common goal of expanding labeled datasets, while enhancing the quality of labels.