Case Study 1 : Digital Gaming and Interactive Technologies for Neurological Care Centres
Collaboration with Industry Partner – Sue Ryder
- Supervisors: Dr Paul Cairns (York), Dr Christopher Power (York), Dr Jeremy Gow (Goldsmiths)
IGGI CDT student Jen Beeston studied Environmental Science at the University of York. She went on to work as an environmental researcher for the Environment Agency and then as a quality and process technician in a manufacturing company. In this last position, she developed refinements of existing processes that led to annual savings for the company of over £500k.
These roles required her to develop research and analytical skills, which together with her passion for digital gaming have prepared her well for the IGGI PhD programme, despite what might be considered a non-traditional route into academic research.
Jen’s PhD was set up in collaboration with Sue Ryder. Though Sue Ryder is best known for hospice care, it also runs several neurological care homes where people who are severely affected by neurological disorders or injuries live with the support and specialist care that they need. Sue Ryder was keen to work with Jen to explore the use of digital games to improve the well-being and quality of life of their service users. Though digital games have previously been considered as ways of improving rehabilitation, the focus for Sue Ryder was not on this because their service users, on the whole, have chronic and degenerative conditions.
Their interest in digital games was to provide a stimulating and motivating activity for their service users that may also allow them to interact with people outside of the care home. The research aim of Jen’s PhD is therefore how best to use games in this context and how to make usable and effective resources for Sue Ryder to deliver the benefits which games can bring across all of their care homes.
The flexibility of the IGGI programme and the requirement to be engaged with end user organisations has allowed Jen to get heavily involved with Sue Ryder, specifically at the Holme Hall home close to York. Jen volunteers there for one day a week in the play therapy sessions. Through this she has already begun to explore the use of existing games and games technology to help some of the service users enjoy playing digital games despite their disabilities. For example, she has been trying out various “one-touch” games on iPads and touch screens. She has also worked to make Holme Hall’s existing IT equipment more suitable for playing digital games.
In her research, she has looked at the gaming community’s perception of aids, like content skipping, that might support disabled players to play socially but that may also be viewed as cheating in players without disabilities. Interestingly, depending on this framing, the gaming community can see content skipping as both a benefit to some players, but also a threat to a notion of being a gamer. She has a paper in preparation on her findings to date.
The IGGI programme so far has developed her skills for research into games including the development and adaptation of games that may be necessary for her research goals. She has had the opportunity to present her work to a games industry audience through the IGGI Symposium and this is also valuable for Sue Ryder who would otherwise been unlikely to have the opportunity to reach this audience.
The large number of students on the IGGI programme are also providing Jen with a goodcommunity of gamers and games researchers that can inform her exploration of games in a challenging context. Additionally, her work is broadening the horizon of the other IGGI students and showing them some of the potential reach of their work.
The next steps in Jen’s research are to look at how players and carers can find games suitable for people with disabilities and also to do some in-depth ethnographic case studies of gamers at Holme Hall to understand the personal, organisational and contextual factors that are important to her research. With her goal to facilitate gaming for residents in neurological care homes, she hopes to contribute and extend existing guidelines for developing and modding games to be more accessible. She also aims to inform
Case Study 2: The application of Self Theory in Digital Games to develop adaptive gameplay models, mechanics and narrative
Industry Partner: Ubisoft Montreal (under negotiation)
- Supervisors: Prof Jonathan Freeman (Goldsmiths), Prof Richard Bartle (Essex), Dr Jeremy Gow (Goldsmiths)
Zoë O’Shea, IGGI CDT student at Goldsmiths (2015 cohort) grew up with a passion for video games, and moved from Dublin to the UK in 2007 to do a BA (Hons) in Computer Games and Visual Effects at Cambridge School of Art, Anglia Ruskin University, followed by a Master’s in Digital Game Theory and Design from Brunel University London. She then worked as the Visual Director and Lead Artist at London-based indie company Tea-Powered Games on their debut title, “Dialogue: A Writer’s Story”.
As both a student and a practitioner of video game design, she became interested in how games interact with individual players’ personalities, and in the gap between designers’ theoretical understanding of games and players (ludology) and the practical concerns of video game development. After watching Ubisoft Montreal Creative Director Jason VandenBerghe’s GDC talk “The Five Domains of Play” on applying the standard ‘Big Five’ OCEAN personality model to predicting player game preferences, Zoë was inspired to further explore her ideas and the IGGI programme offered the perfect package to justify her move back to academia from her developing career in the games industry. Her thesis aims to build on this work, extend it into areas of identity, motivation and emotional response (“self-theory”), and investigate its practical use in game development. Her topic synthesises the knowledge and experience of her previous work to take an interdisciplinary approach utilising game design, ludology, psychology and industry practice.
The IGGI CDT training programme has helped Zoë further develop the skills she needs for her PhD. The research skills module provided a solid grounding in a range of relevant quantitative and qualitative techniques. The game development module has strengthened her Unity skills, needed to build custom games for empirical studies with players. The game design module lead at Essex, Richard Bartle, is one of pioneering researchers in the field of ludology and is internationally recognised for his work regarding the understanding of the motivations and personalities of game players. As a consequence of taking the module, Zoë had the opportunity to discuss her ideas with Richard extensively, and recruit him as her second supervisor early on in her PhD. Beyond the formal programme, Zoë had the opportunity to attend Develop, the leading UK games industry conference.
Zoë is passionate about encouraging girls to pursue technical careers in computer science and games design and development. To this end she has been working with two other IGGI students: Tara Collingwoode-Williams and Tom Cole (a former teacher) to present at local schools and contribute to the “Women in Computing” project at Goldsmiths.
When asked what being an IGGI student means to her, she replied:
“I feel like I am part of a new movement in games where passionate people can take their skills, develop and create new and exciting projects to make the world a better place. I am part of something greater than myself, advancing games as part of a community, in whatever form that might take, while still maintaining my agency, individual perspective and processes. I love the cohorts and the sheer variety of ways that IGGI has connected people together. If I am ever stuck or in a rut, I know that all I have to do is seek out another IGGI member and speak with them for inspiration. I am constantly amazed by the opportunities IGGI provides and opens for us in academia, development and industry. The help and mentorship of my supervisors and other IGGI lecturers has been invaluable in improving my personal development and research.”
As part of her IGGI programme, Zoë wishes to pursue an industrial placement with Ubisoft Montreal. Being a member of the IGGI research centre allowed for her to directly contact Ubisoft Montreal Creative Director Jason VandenBerghe with confidence. As a result, discussions are already in progress to have Zoë travel to the large development studio and collaborate with the individual who inspired her current research.
On her future ambitions, Zoë says “I would love to make games that encompass everything that I am learning about player psychology and design, perhaps by going into consultancy for games companies like my supervisor Richard Bartle. I’m passionate about teaching a new generation of game designers how to understand players.”
Case Study 3: AutoGraff: A Procedural Model of Graffiti Form
Industry Partner: Media Molecule
- Supervisors: Prof Frederic Fol Leymarie (Goldsmiths), Prof Simon Lucas (Essex)
Virtual cities in games, such as Los Santos in Grand Theft Auto 5, are growing larger and more intricate, and there is a demand for greater realism. Graffiti and street art are an essential feature of the modern urbanscape. Procedural graffiti tools would allow designers to create more realistic urban environments in less time, just as there are already tools for generating trees and vegetation. It also opens up possibilities for more dynamic urban environments, where text generated in-game could be rendered as graffiti.
Daniel Berio’s (2014 cohort, Goldsmiths) CDT research focuses on modelling the perceptual and dynamic processes involved in the production of graffiti art, developing a system capable of procedurally generating realistic custom graffiti within urban game environments.
Daniel came to the IGGI programme with strong background in technology and the creative industries. A graffiti artist since the age of 14, he has worked as a graphic designer, video game developer and creative coder. During his Masters degree at the Royal Academy of Art, The Hague (the Netherlands) he began to develop an interest in generative systems, and how the complex aesthetics of graffiti could be produced algorithmically.
Daniel has proposed a “movement centric” approach to the procedural generation of synthetic graffiti, in which stylised curves, shapes and patterns are generated via a simulation of the (human) movement underlying their production. This has lead to the creation of a flexible image generation tool, that captures the variability that is typical of hand-drawn traces, as well as the generation of smooth and natural motions that can be used to generate realistic animations and to transfer drawing skills to embodied agents, such as non-player characters (NPCs) in games and physical robots. Daniel has published this work in the ACM Eurographics Expressive Graphics Symposium 2015 , where he was awarded “Best Presentation”, and the IEEE/RSJ International Conference on Intelligent Robots and Systems 2016 . Two further submissions are currently under review, including a collaboration with fellow IGGI student Memo Akten , applying deep learning to movement-centric generative model of handwriting, which has the potential for significant commercial impact through greatly compressing the data needed to record handwritten text, and to create handwritten text in an individual style from handwriting examples.
The project has led to some ongoing scientific collaborations between Daniel, his principal supervisor Prof Fol Leymarie, and a number of international researchers:
- Prof Réjean Plamondon (Ecole Polytechnique, Montreal, Canada). Daniel is using mathematical models Prof. Plamondon originally developed for biometric/ handwriting analysis, who is keen to see new creative applications of his work.
- Prof. Johan Wagemans (KU Leuven, Belgium). Daniel’s work provides a new domain for the interests of Prof. Wagemans in psychophysics: studying the visual perception of abstracted forms (e.g. handwritten letters) through computational models, understanding the impact of movement qualities on the perception of static forms, and studying the effect of robotic embodiment on the perceived quality of computationally generated forms.
- Dr Sylvain Calinon (Idiap Research Institute, Switzerland). Daniel and Dr. Calinon have worked closely on developing a graffiti-drawing robot based on Daniel’s work, and on how movement synthesis techniques that are more typically used in robotics (e.g. Model Predictive Control) can be imported into computer graphics applications.
Although only just over two years into his PhD, there are already scientific impacts of Daniel’s work in the form of cross-fertilisation of movement-based models between domains. Prof. Plamondon’s work from handwriting analysis was not previously known in robotics, and this has initiated a research link between Prof. Plamondon and Dr. Calinon.
To further the impact of this work on the robotics community, Daniel, Prof. Fol Leymarie and Dr. Calinon organised an international workshop on “Artistically Skilled Robots” at the IEEE Conference on Intelligent Robots and Systems in October 2016. Consisting of 9 invited talks over one day, including speakers from Google and Microsoft Research, with 50 participants it was the best attended of 18 parallel workshops and tutorials. Following on from the workshop, a special journal issue of extended contributions from workshop participants is currently being assembled, and the organisers are in discussion with a publisher to edit a book on this topic. A follow-up workshop is to be held in 2018, in affiliation with major annual robotics conference, ICRA.
As a next step, he plans to: continue his exploration of the relationships between visual qualities of graffiti traces and the corresponding drawing-movement dynamics; extend his work to increasingly more complex forms of graffiti letter stylisation; and provide novel ways of generating synthetic graffiti and calligraphy through the interactive and procedural manipulation of motion paths. His vision is that movement centric approaches to image generation will be useful for the computer graphics community at large, and he plans to submit to major computer graphics conferences such as SIGGRAPH and Eurographics. There is also the opportunity for this kind of computational study to bring new insights into the complex cognitive, perceptual and motor processes involved in the production of graffiti art, as well as other related art forms, with further impacts in robotics, human perception understanding, creativity modelling, and feedback in the art practices of game design (creative industries), graffiti making (street art) and, more generally, handwriting and calligraphy.
Daniel’s work was exhibited to the 22,000 attendees of the New Scientist Live exhibition  and reported in the press .
-  Project website: http://www.doc.gold.ac.uk/autograff/
-  D. Berio, D. & F.F. Leymarie, “Computational Models for the Analysis and Synthesis of Graffiti Tag Strokes” in Expressive Graphics Symposium / ACM Eurographics, June 2015.
- Received the *Best Presentation Prize* : http://expressive.richardt.name/2015/Prizes
- Available with video: http://www.doc.gold.ac.uk/autograff/wp/?p=199
-  D. Berio, S. Calinon & F. F. Leymarie, “Learning dynamic graffiti strokes with a compliant robot” in Proc. of IEEE/RSJ Intl Conf. on Intelligent Robots and Systems (IROS). Daejeon, Korea, October 2016. Available with video: http://research.gold.ac.uk/18857/
-  M. Atken, D. Berio, F. F. Leymarie, M. Grierson, and R. Plamondon (2017) “Sequence Generation with a Physiologically Plausible Model of Handwriting and Recurrent Mixture Density Networks”, submitted to the 5th International Conference on Learning Representations, Toulon, France.
-  Graffitizer demo exhibited at New Scientist Live, 28 September – 1 October 2016: https://live.newscientist.com/graffitizer-daniel-berio/
-  http://www.techradar.com/news/robot-graffiti-artist-is-the-banksy-of-the-mechanical-world.