The challenge
The University receives a large influx of free text comments in the form of feedback from students throughout the year, including data from surveys like the National Student Survey (NSS) and module evaluations. With this constant stream of free text comments, many departments struggle to efficiently analyse all the data and respond in a timely manner. Staff, across many departments, often spend days and weeks manually reading and analysing free text comments to understand concerns and areas for improvement.
To address this challenge the IT Innovation Team have developed a new feedback analysis tool. This tool aims to help staff through the analysis process by using Artificial Intelligence (AI) to read, organise, and understand free text comments. The tool’s central aim is to provide understanding and insights quickly to help better inform decision-making.
The technology
The IT Innovation Team have developed a user-friendly web app that simplifies the process of analysing feedback from any survey. The solution enables people to use very powerful AI through a simple web interface, without needing to understand the complexities of AI. The tool uses a branch of AI called natural language processing to perform entity analysis, key phrase analysis, and sentiment analysis of any content that is uploaded. The technology can read and analyse 250 paragraphs of student feedback in under two minutes. This provides insights and data exports back to users in seconds and minutes, not days and weeks.
Entity analysis helps us identify important elements within the uploaded feedback, such as specific topics, people, or organisations mentioned. This feature enables us to understand the core themes and subjects discussed in the feedback, providing valuable insights for improvement efforts.
Key phrase analysis looks deeper into the feedback to pinpoint significant phrases or terms. By highlighting key phrases we can grasp the essence of what’s being communicated, helping us prioritise areas of concern or focus.
Sentiment analysis is used to identify the emotional tone of the feedback, whether it’s positive, negative, or neutral. This aspect is crucial for understanding the overall sentiment of the feedback, enabling us to distinguish between entities or key phrases that are mentioned in a positive light as opposed to those referenced negatively.
Training AI that is unique to the University
Alongside existing language models, the tool makes use of custom models for language analysis, allowing users to train models on previous responses from University surveys. Already, the tool has enabled the development of several custom models tailored to specific feedback survey responses. For instance, the tool provides an NSS model and a JISC digital insights model. Customised models ensure that the analysis is finely tuned to the specific needs and nuances of the University context. Models are developed by context experts across the University who are responsible for analysis.
Technical architecture
Lessons learned
Conversations with library staff shed light on an unexpected benefit of the tool. Beyond saving time, one staff member highlighted its potential to protect mental well-being by sparing her pain of having to read negative feedback, while still giving her valuable insights to help fulfil her job effectively. This insight highlights the broader impact AI could have in promoting efficiency and well-being among University staff.
The development provided useful technical insights as to the availability and integration of powerful AI, through existing technology partners at the University. Through cloud computing it is now possible for the University to access very powerful AI at relatively low cost. From a service design perspective, it’s now also possible to implement new AI features into existing services through relatively simple Application Programming Interface integration. Tools such as Microsoft Azure Language Studio are good examples of the types of technologies that are becoming readily available.
Additionally, it was important to recognise the feedback analysis tool as an augmentation, not a replacement, of existing staff and processes. This underscores the need to emphasise collaboration and the complementary role of AI in enhancing rather than attempting to supplant expertise.
Other use cases
It’s likely this technology has applications beyond just the understanding of survey responses. At its core the technology can read and understand any form of free text. Whilst specifically designed to aid survey understanding the technology likely has other applications. It could be used when staff need to analyse large volumes of text to extract useful information, speeding up and streamlining the process, which then quickly provides University service owners with insights and understanding.
Future prototypes and discoveries
The IT Innovation Team is always looking for challenges that add value to the University. If you have an idea for the development of a technology that supports teaching, learning and related administration across the University, please get in touch with the team. We are always looking for new and interesting problems to solve.