Shadowpox in Toronto

Technical director LaLaine Ulit-Destajo, Theatre@York technician James McQuay and I brought the Shadowpox project to its North American debut this week at a symposium hosted by the Art and Science of Immunization working group at the University of Toronto's Jackman Humanities Institute, in conjunction with Public Health Ontario.

Public Health Ontario published an in-depth review, "Building Bridges: The Art and Science of Immunization Symposium":

This event explored the breadth of perspectives on immunization issues and how those in arts and science fields can work together to address them. A major focus of the day was on vaccine hesitancy. There was significant discussion around how a humanities approach can inform how we communicate with the public and weave a narrative to help bridge the gap between science and the public. Attendees were a diverse mix, with backgrounds in fields such as immunology, epidemiology, history, English, anthropology and even theatre studies.... 

Allison Humphrey, a PhD student in Cinema and Media Arts from York University, introduced her interactive motion capture game, Poxémon. Players users use their arms and legs to fight off the “shadowpox” disease that attacks the individual, meanwhile trying to protect 100 other characters on the screen from infection. This game helps players to understand how infectious diseases can spread in a population, or be protected by vaccination. Attendees had the opportunity to demo the game during breaks.

Their video even includes footage (at 0:40) of Shadowpox in action:

Play #YorkU PhD candidate @alisonhum's #Shadowpox #game at #ArtSciImmunize symposium Apr 13 https://t.co/p1VfjPcWCi #ImmuneNations pic.twitter.com/M7nAXSXM3s

— YorkuAMPD (@YorkuAMPD) April 12, 2017

Up a ladder without a paddle. @alisonhum #immuneNations #ArtSciImmunize pic.twitter.com/J2pUdkqOO4

— Lalaine Ulit Destajo (@ludestajo) April 13, 2017

Practice fighting off vaccine preventable disease with this interactive game at #ArtSciImmunize! pic.twitter.com/KCaCtrEsFp

— Public Health ON (@PublicHealthON) April 13, 2017

Read the full post for more Tweets from the symposium...

Shadowpox in Trondheim

Shadowpox made its debut in Trondheim, Norway, at <Immune Nations>, an evidence-based art exhibition about the constructive role that art can play in global political discourse around life-saving vaccines.

Annemarie Hou gets a 100 protection score!

Shadowpox talk for students from the Trondheim Academy of Fine Art

SciArt Magazine covered the opening at Galleri KiT: "The Art of Vaccination" (The Lancet reviewed the opening in Geneva).

Syringe Sequence #1-2, Kaisu Koski

See more photos below...

Shadowpox in the Lab

Shadowpox at RADA: Abraham Popoola, video still, 2016

LaLaine Ulit-Destajo, Caitlin Fisher, Steven Hoffman, Susan Rogers Van Katwyk and I are in the final weeks of preparation for a gallery installation incarnation of Shadowpox as part of the <Immune Nations> exhibition in Trondheim, Norway, and Geneva, Switzerland.

<Immune Nations> is an experimental evidence-based artistic-research exhibition emerging from the three-year Vaccine Project, part of the International Collaboration for Capitalizing on Cost-Effective and Life-Saving Commodities (i4C) funded through the Research Council of Norway's GLOBVAC Programme.

The exhibition will kick off at Gallery KiT on March 13, as the official opening of the 10th Conference on Global Health and Vaccination Research. From Trondheim it travels to UNAIDS in Geneva on May 23 for the 70th World Health Assembly. 

Here are a few work-in-progress photos from the Alice Lab for Computational Worldmaking and the Augmented Reality Lab at York University:

Using Rulr to calibrate the Kinect and projector

LaLaine's motion-capture stick-figure animation in development (Bézier curves are on the way!)

LaLaine's openFrameworks Shadowpox code

LaLaine infects the lab with sprites

Title animator Jos Humphrey gives a peek behind the curtain in After Effects

Jos's title sequence on the big screen

Alison and Maggie the Mannequin model the latest in VR headsets, but not because they're part of the project.
They just make us look suave.

Mo' Better Mocap Shakespeare

The Royal Shakespeare Company's performance-capture production of The Tempest, in collaboration with The Imaginarium and Intel, is running a series of talks including one this weekend that I would sorely like to see: Creating The Tempest: Brave New Digital World.

The strapline reads, "What does the future hold for digital storytelling in theatre?" and the speakers include the visionary Sarah Ellis, RSC Head of Digital Development. I had a chance to talk with Sarah this summer about the breathtakingly ambitious production, and all the brain-bending, tech-twisting challenges that go into animating in real time, live on stage.

Mark Quartley as Ariel and Simon Russell Beale as Prospero (photo: Topher McGrillis)

I'm hoping I'll also have a chance to see the final show in person, even if the cinema broadcast makes it to Toronto, because one of those many vexed questions is what surfaces and substances to project the avatar onto once you've cooked it up in the computer.

Judging by the production photos, it looks like the RSC's answer is: "Everything that's nailed down, and maybe a few people who aren't." So it would be a shame to have to squint at that 360-degree experience through the keyhole of a movie screen.

The Company of The Tempest (photo: Topher McGrillis)

But in a way, watching a theatre actor... tracked by infrared motion-capture cameras and inertial sensors... driving real-time digital animation... projected onto a cylindrical scrim and a thrust stage... all recaptured on HD cameras... and broadcast into cinemas around the world? That's no more meta than watching a male actor play the female Rosalind pretending to be the male Ganymede role-playing "Rosalind" so that Orlando can hone his wooing chops.

Simon Russell Beale as Prospero and Mark Quartley as Ariel (photo: Topher McGrillis)

Viral Vanier

So on the very same day the Vanier Canada Graduate Scholarships are announced, I learn that not one but two of my PhD classmates in Cinema and Media Studies at York are through to the next round for next year. Go, Claudia and David!

And congratulations, Zachary, Jesse and Syrus Marcus!! Reading about their work, and the research abstracts for the 162 other new Vanier Scholars, feels like looking at one of those gorgeous rainbow MRI scans of Canada's brain...

 Henrietta Howells, NatBrainLab, Wellcome Images (Creative Commons by-nc-nd 4.0)

Live Cello Mocap and Projection Mapping

59 Productions have some wicked plans in store for projection mapping in live performance, but here's Sol Gabetta performing Elgar's Cello Concerto in E minor at the BBC Proms, as an example of what they can already do:


(h/t to Liz Barber)

The video below shows the how. It highlights four key challenges with motion capture and projection mapping in live performance.

Prospero meets Ariel at the Imaginarium

More fascinating glimpses behind the scenes on preparations for the Royal Shakespeare Company's new production of The Tempest: "Simon Russell Beale, playing Prospero, and Mark Quartley, playing Ariel, meet for the first time, coming together with Imaginarium crew and Intel technology to explore how Ariel's avatar will work."

Performance Capture for Stage, Film and Games

A round-up of links for further exploration of performance capture, offered with respect and gratitude to the massively talented graduating RADA actors who participated in the first-ever Shadowpox workshop – Fehinti Balogun, Natasha Cowley, Sayre Fox, Skye Hallam, Tom Martin, Polly Misch, Abraham Popoola, Maisie Robinson and Jamael Westman.

Science + Fiction + Theatre

[This post was originally published in March 2015, but I'm reposting it because it's good background for a workshop I'll be doing next week. As collaborator Simon Eves of PLASTIK pointed out recently, there's still not enough sci-fi onstage...]

At a recent conference on public health history at the University of Toronto, I had some intriguing conversations about crossovers between science and art/entertainment, particularly how science fiction can welcome audiences deep into issues in public health.

It brought back a chat I had last year with Conall Watson of the Department of Infectious Disease Epidemiology at the London School of Hygiene and Tropical Medicine.

He described his work on a promenade theatre event called Deadinburgh, set during a zombie outbreak besieging the Scottish capital.


His team ran the public health advisory cell, "tasked with guiding the audience through different approaches to controlling the zombie epidemic; giving them insight into the usually back-of-house practices of the public health authorities.

We also had input into the epidemiological parameters and narrative of the overall show."

In a paper titled "Deadinburgh: zombie epidemics, citizen power and public health", he and his colleagues Kate Harvey and Nigel Field of University College London, describe the scenario:

"An unknown pathogen was ravaging Scotland’s capital in April 2013, turning unlucky infected souls into bloodthirsty, ambling beasts. The city was under military lock-down and scientists were working around the clock to identify the pathogen and develop means of control.

"Each night, 250 uninfected citizens reached the safe zone at a former veterinary college, taking democratic responsibility for the public health and military response.

"Whether immersive theatre and simulated situations can get people to engage with public health on a larger scale and help build trust and empathy with the way that science is used to inform public sector decision-making remains to be seen. What we do know is that people like science; people like zombies; and the two combined can help us to reflect on our own practice as public health professionals."

In the video below, Kate Harvey says, "Bringing in something from popular culture helps to appeal to a wider audience.... Public health has both art and science at its core. Public health is the art and science of promoting health and preventing disease and prolonging life.... But maybe what we haven't done so much of is using it as a means of communication, and actually putting some of the science back into art as well."

Click here to watch a video of Conall Watson and Kate Harvey discussing "Deadinburgh - the science of zombies" – a London School of Hygiene and Tropical Medicine podcast.

So long, green screen

As I work with a Kinect, a projector and a digital video camera to drive and record real-time interactive projected effects, I'm already fantasizing about hacking the Lytro Cinema system together with one o' these.

Come on, it's only a $125,000 rental...



(H/t Jos Humphrey)

The Invisible Pantry

Created for Jennifer Jenson's Cultural Studies of Educational Technology course.