Generative AI in Education: Framework and Ethical Considerations for Surveys

In their May 2, 2025 paper titled Methodological Foundations for AI-Driven Survey Question Generation, Ted K. Mburu and co-authors present a framework using large language models to generate adaptive survey questions for educational research, while addressing ethical concerns such as bias and privacy.

This paper introduces a framework for using generative AI in educational surveys, focusing on adaptive question generation and iterative refinement through the Synthetic Question-Response Analysis (SQRA) method. Using Activity Theory, it examines how AI tools influence participant engagement and learning while addressing ethical concerns like bias and privacy. Evaluations of AI-to-AI and AI-to-human interactions reveal the potential and limitations of AI-driven surveys, emphasizing the importance of robust prompt engineering for reliable data collection in education.

In an era where artificial intelligence (AI) increasingly permeates daily life, understanding how humans interact with large language models (LLMs) has become a pressing concern. While LLMs excel at generating coherent responses, their ability to replicate the subtleties of human conversation remains a subject of intense scrutiny. This study delves into the nuances of human-AI dialogue, examining alignment, coherence, and emotional engagement. By analysing interactions from controlled experiments, researchers sought to determine how closely AI can mirror human communication while fostering meaningful connections.

The Study’s Approach

To explore these dynamics, the research compared conversations between humans and those involving LLMs. Discussions spanned various topics, including problem-solving, emotional support, and casual dialogue. Alignment was measured using computational metrics like cosine similarity, assessing how well AI responses matched human expectations. Emotional engagement was evaluated through sentiment analysis and keyword frequency, providing insights into the depth of connection.

The findings reveal that while LLMs generate coherent and contextually relevant responses, they often miss the nuanced emotional subtleties inherent in human conversation. For instance, when discussing sensitive topics, participants reported feeling less emotionally supported by AI compared to humans. Statistically, interactions with LLMs achieved 82% alignment with human expectations, significantly lower than the 95% observed in human-to-human conversations.

These results highlight the need for a balanced approach in AI design, combining technical precision with emotional intelligence. As LLMs integrate into everyday communication tools, developers must prioritise empathy alongside accuracy. The study suggests that hybrid models, blending rule-based systems with machine learning, could offer a promising solution.

This research underscores the potential and limitations of large language models in conversational settings. While they demonstrate remarkable capabilities as communication tools, their effectiveness hinges on replicating not just words but the human touch. As AI evolves, understanding these nuances will be crucial for creating systems that enhance interactions meaningfully rather than merely imitate them.

The findings provide a foundation for future advancements in conversational AI, focusing on measurable metrics and real-world applications. By addressing the emotional gap, developers can create more intuitive and supportive AI tools, ultimately enriching human-AI interactions.