Cardiff graduate's success with software for autistic learners

In his final year Joseph Liu worked with Cardiff-based teachers to develop software that could change the way autistic learners communicate in the classroom.

While he was growing up, Joseph, who is autistic, used PECS ('Picture Exchange Communication System') in the classroom.

PECS is a system used to help mitigate communication challenges for children with Autism Spectrum Disorder (ASD), and is usually a book containing images that is used to communicate with parents, teachers and other adults without being verbally communicative.

For his final year BSc Computer Science undergraduate project, Joseph was motivated by the idea of making a more customisable version of the system suitable for both learner and teacher users in a classroom environment.

As part of his project, Joseph had a real-life client who he aimed his software at. He worked with teachers in Pontprennau Primary School and Pontprennau Nursery, who tested the system and gave feedback while it was in its development stage.

The main difference between Joseph's 'PECSOnline' and the more traditional version of PECS is that Joseph designed his system to be fully personalised.

For example, if the user wants to talk about a jacket, users can take a photo of their own jacket and use that as the symbol for ‘jacket’ (as opposed to a cartoon jacket, for example).

"I recognised how important reviews are, and getting the target features are to the client so that I would know what they want to have,” said Joseph.

“I added a passcode for changing items and accounts to stop unauthorised changes, as well as public and private personalised symbols. I also made a video demonstration of the application so that clients could see all of the features for themselves.”

Following the project, Joseph and his supervisor, Dr Fernando Lozides, were able to publish their work and share internationally.  They co-authored a peer-reviewed paper in 'Human-Computer Interaction – INTERACT 2023', and Joseph subsequently presented to the INTERACT 2023 conference in York last September.

The international conference, which is held every two years, centres around human-computer interaction.

“It was a really nice experience and I got to speak to loads of people. It gave me the confidence to realise I’m capable of doing a PhD thesis,” said Joseph.

Joseph explained that he will continue to work on PECSOnline now that he’s graduated, and that he would be keen to find clients to help develop it. He plans to return to Cardiff in the future to complete a PhD and expand his work on the system.

Dr Fernando Lozides said: “We received the requirements from a real project, a real need, in school. Joe fulfilled that and built this PECS application and then we tested it with the teachers there who tested it in the classroom and gave us feedback to improve it.

It’s a great achievement for an undergraduate to be published, and it’s something that’s very rare. Usually somebody who is doing their PhD or their masters perhaps will think about submitting something but for somebody who is an undergraduate, Joe did phenomenally well.“