RESEARCH METHODS
Comparative Research
I decided to proceed with a comparative research approach with a semi-structured interview with four tutors from the MA Design Management Course.
I read ‘Teaching Comparative Thinking to Strengthen Student Learning’ (Silver, 2010) which outlines that comparative thinking is ‘one of our first and most natural modes of thought’ and therefore I felt would work effectively alongside a semi-structured interview which is flexible and exploratory and enables movement within a topic.
Most of the reading I could find on comparative methods tended to focus on social sciences and communication science which are not directly related to my topic however the principles of prompting conversations and observations and exploring perspectives through the mode of comparison was interesting to me.
Silver, H.F. (2010) Compare & Contrast: Teaching comparative thinking to strengthen student learning. Alexandria, VA: Association for Supervision and Curriculum Development.
The comparative method was as follows:
- Use ChatGPT to generate feedback on a previously marked assignment by the tutor
- Input a student assignment and also the assessment criteria
- Ask the tutor to read the ChatGPT-generated feedback and discuss
- Ask the tutor to then compare the ChatGPT-generated feedback to their original feedback
- Semi-structured interview to discuss this comparison
Self Ethnography
I also conducted a comparative marking exercise on my own assignment for the Inclusive Practices unit for the PGCert. This would give me the opportunity to reflect on how I, as a student, felt about the comments from ChatGPT on my own work.
This type of research enables the reseracher to ‘engage in rigorous self-reflection—typically referred to as “reflexivity”—to identify and interrogate
the intersections between the self and social life’
Adams, T.E., Ellis, C. and Jones, S.H. (2017) ‘Autoethnography’, The International Encyclopedia of Communication Research Methods, pp. 1–11. doi:10.1002/9781118901731.iecrm0011.
Why semi-structured interviews?
- As outlined by Mathers et al (2000) “With semi-structured interviewing, the open-ended nature of the question defines the topic under investigation, but also provides opportunities for the interviewer and interviewee to discuss some topics in more detail“
- This approach would also be defined as Interactive Rationalism where the researcher “embraces ‘soft’ and flexible technical measures in dealing with the problem of how to maximize reliable responses. Techniques include continually sharing any emerging interpretations and insights with those studied, conducting interviews of various kinds and in various places” (Alvesson)
- The interviewees were all people I have existing work relationships with, I know them relatively well and could adopt an informal conversational style of interview. The potential downside of this is they may also feel less comfortable being open knowing that they are
- Secondary Research & Other sources
My references and bibliography are situated in a different post (ARP | references) and my secondary research predominately focused on:
(1) understanding the current landscape of AI in Higher Education and the arguments and positions around that.
(2) the principles of the research methods
(3) issues around the fairness of marking and feedback, again this is a vast topic, and therefore tried to limit my reading to
I attended several Webinars from inside and outside UAL including a Symposium on the use of AI in Higher Education
I have approached Digital Learning team within UAL on what projects or workstreams are exploring around the use of AI for teaching staff. I have found the University focus tends to be on the use of AI for students and it is less clear what work or research is being done for teaching.
Limitations and reflections:
Size of sample: I recognise this was a relatively small sample of tutors to work with, however, each interview was an hour in length (if not a little more), and due to time considerations I felt this was achievable. The analysis phase was going to take place at the same time as I had a significant Final Major Project marking to undertake for my MA course and it was important for me to manage my workload over the December period.
In terms of Action Research, I felt that this intervention was sufficient to explore initial thoughts and ideas from the teaching staff and due to the size and scope of this area of study (AI in Higher Education) I could see that there were many ways it could spiral and keeping the study small and compact felt like the most appropriate starting point. This was reinforced in our PG Cert Workshops which reassured me.
I would have loved to have included a student perspective and shared the feedback from ChatGPT and tutors with the students whose work I was using as the sample but again, they were in the middle of completing their FMPs and it would have been unreasonable to request their time for this study.
However, I do feel there is a huge scope to move this project to another phase, and may look to do this in the future if time and resources allow.
I had some set questions for the semi-structured interviews which worked well however it quickly became evident that tutors were keen to do more exploration around the capability of ChatGPT and explore within those topics to ‘test’ the technology as shown below, In general this worked effectively and allowed us to explore and let the tutor interest and concerns lead the discussion