MethodMentor (School of Education/CHARM-EU)

The motivation for this project emerged from the experiences of MSc and PhD supervision and Research Methods teaching by the proposers across disciplinary contexts (School of Education, School of Computer Science, School of Natural Sciences). From working with a diverse range of students (i.e. Research Methods experience, prior disciplinary background, ability levels) the need for more personalised support to reflect, compare, and select appropriate Research Methods was identified, especially in areas where mixed methods are common.

Unequitable student experiences in Research Methods learning are common. While some students had prior knowledge and experience of Research Methods, others had limited experience. Others struggled with Research Methods outside the disciplinary scope of their undergraduate studies. Challenges with Research Methods learning is supported by literature indicating that students often have negative perceptions of Research Methods (Breit and Liu, 2013), face difficulties with understanding concepts (Stoa, Chu and Gurung (2020), lack motivation (Earley, 2013), and require more personalized support (Rich, 2014).

The proposers are thus motivated to better support and provide a more equitable student experience in Research Methods modules. The proposers have a background in Computer Science and Education and an interest in exploring Artificial Intelligence (AI) innovations for education and this was a further motivational factor.

Project relevance

• Institutional relevance: Relevant to Trinity’s Research Excellence Strategy goal 6 and to Trinity teaching due to the high prominence of Research Methods in MSc/PhD curricula, with 147 cross-disciplinary modules identified on Blackboard.
• Pedagogical relevance: Indications from the literature to develop pedagogical innovations to support students in Research Methods space (Mitchell and Rich, 2020).
• Technological relevance: Indications from the literature that pedagogical development needs to explore AI possibilities (Díaz and Nussbaum, 2024).
• Inclusivity relevance: Provides the ability to personalize and better support students embarking on research projects and may bridge equity gaps across students.

• Develop an AI chatbot to support MSc/PhD student knowledge, critical evaluation, reflection, comparison, and selection of Research Methods.
• Scaffold conversation design using best practices of educational chatbots, dialogic pedagogy and co-creation with students and supervisors.
• Leverage open access materials to support knowledge retrieval within the field of Education, focusing on qualitative, quantitative and mixed methods.
• Deliver the AI chatbot to 20 MSc/PhD students within the School of Education, CHARM-EU and other interested supervisors
• Evaluate the AI chatbot, its impact, perceptions, scalability, and usefulness both from a student and teacher perspective using surveys and interviews.
• Communicate results via academic and practice based publications.
• Provide a model for wider scalability across the university.

Phase 1: October 2025 – February 2026

Technical Design

• Draft technical requirements document (e.g. users, functional requirements, data requirements, data architecture) including platform evaluation criteria.
• SWOT analysis of technical platforms including data protection considerations
• Discussions with IT Services for insights into data protection and system integration with Trinity IT services.
• Exploratory prototyping across multiple potential AI platforms 

Educational Design

• Narrative review of educational chatbots, dialogic pedagogy, and Research Methods supervisory practices. 
• Identification, evaluation, and retrieval of open access published Research Methods literature for RAG.
• Informal interviews with MSc/PhD Supervisors from target disciplines to inform how they support students in selecting Research Methods. 
• Informal interviews with MSc/PhD Students from target disciplines to inform how they would like to be supported in selecting Research Methods.
• Analysis of interviews with MSc/PhD Supervisors and Students from target disciplines to inform chatbot dialogue patterns.
• Discussions with DEI Office on inclusivity requirements
• Selection of conversation design framework
• Identification of student personalization levels
• Learning objective definitions 

Implementation

• Recruitment and commencement of Research Assistant (either Phase 1 or 2 where possible)
• Placeholder invitation to participate in pilot sent to students and supervisors
• Ethical approval sought from School of Education for research

Phase 2: February 2026 – June 2026

Technical Design

• Informed selection of technical platform based on Phase 1 results
• Technical prototyping including conversation implementation, RAG, logging and monitoring system, red flag and security modelling, and reporting back end including accuracy and efficiency monitoring.
• Build knowledge base using open access Research Methods literature for RAG
• UX testing of initial prototype with proposal team
• Integration of evaluation feedback into platform
• Develop technical rubric for quality assurance of chatbot performance

Educational Design

• Initial conversation design based on narrative review and interviews.
• Presentation of initial conversation design to previously interviewed Students and Supervisors for feedback
• Refinement of learning objectives

Implementation

• Invitation to participate in pilot sent to students and supervisors
• Develop evaluation materials (e.g. surveys and interview schedule)

 Phase 3: July 2026 – December 2026

Implementation

• First pilot for 10 students in School of Education and CHARM-EU 
• Collection of evaluation data on first pilots

Technical Design

• Refining and updating platform

Educational Design

• Refine conversation flow based on feedback and logs

 Phase 4:  January 2027 – March 2027

Implementation

• Second pilot for 10 students in School of Education and CHARM-EU. 
• Collection of evaluation data on second pilots
• Evaluation analysis and reporting
• Scalability analysis based on evaluation data
• Contacting Research Methods module coordinators across Trinity for workshop participation.
• Delivery of three workshops with research methods module coordinators and related support departments across Trinity presenting the tool and gaining feedback on disciplinary specific needs for scalability possibilities.

Technical Design

• Refining and updating platform

Educational Design

• Refine conversation flow based on feedback and logs

Deliverables

• Deliverable A: Narrative review of educational chatbots, dialogic pedagogy, and Research Methods supervisory practices to inform conversation design. 
• Deliverable B: SWOT analysis of shortlisted AI platforms for platform and data governance selection.
• Deliverable C: Technical requirements and specification document for AI chatbot including TCD systems requirements.
• Deliverable D: A functional prototype delivered to 20 MSc/PhD students from the School of Education including backend and data monitoring requirements. 
• Deliverable E: User guide and workshop materials for further dissemination across Schools.
• Deliverable F: Hosting of three online workshops with Research Methods module coordinators for scalability across Trinity.
• Deliverable G: Final evaluation and scalability report including student and staff feedback, technical performance, and future scalability and refinement needs.

Benefits

The proposed project has the following benefits:

• Addresses Trinity research and teaching strategic goals
• Contributes to the research space in both AI and Education, and Research Methods teaching.
• Post-piloting can have large scale dissemination and impact across Trinity Schools delivering Research Methods modules.