Tuesday, December 13, 2016

Mo' Better Mocap Shakespeare



The Royal Shakespeare Company's performance-capture production of The Tempestin collaboration with The Imaginarium and Intel, is running a series of talks including one this weekend that I would sorely like to see: Creating The Tempest: Brave New Digital World.

The strapline reads, "What does the future hold for digital storytelling in theatre?" and the speakers include the visionary Sarah Ellis, RSC Head of Digital Development. I had a chance to talk with Sarah this summer about the breathtakingly ambitious production, and all the brain-bending, tech-twisting challenges that go into animating in real time, live on stage.

Mark Quartley as Ariel and Simon Russell Beale as Prospero (photo: Topher McGrillis)

I'm hoping I'll also have a chance to see the final show in person, even if the cinema broadcast makes it to Toronto, because one of those many vexed questions is what surfaces and substances to project the avatar onto once you've cooked it up in the computer.

Judging by the production photos, it looks like the RSC's answer is: "Everything that's nailed down, and maybe a few people who aren't." So it would be a shame to have to squint at that 360-degree experience through the keyhole of a movie screen.

The Company of The Tempest (photo: Topher McGrillis)
But in a way, watching a theatre actor... tracked by infrared motion-capture cameras and inertial sensors... driving real-time digital animation... projected onto a cylindrical scrim and a thrust stage... all recaptured on HD cameras... and broadcast into cinemas around the world? That's no more meta than watching a male actor play the female Rosalind pretending to be the male Ganymede role-playing "Rosalind" so that Orlando can hone his wooing chops.

Simon Russell Beale as Prospero and Mark Quartley as Ariel (photo: Topher McGrillis)



Tuesday, October 4, 2016

Viral Vanier

So on the very same day the Vanier Canada Graduate Scholarships are announced, I learn that not one but two of my PhD classmates in Cinema and Media Studies at York are through to the next round for next year. Go, Claudia and David!

And congratulations, Zachary, Jesse and Syrus Marcus!! Reading about their work, and the research abstracts for the 162 other new Vanier Scholars, feels like looking at one of those gorgeous rainbow MRI scans of Canada's brain...

 Henrietta Howells, NatBrainLab, Wellcome Images (Creative Commons by-nc-nd 4.0)

Tuesday, July 19, 2016

Live Cello Mocap and Projection Mapping

59 Productions have some wicked plans in store for projection mapping in live performance, but here's Sol Gabetta performing Elgar's Cello Concerto in E minor at the BBC Proms, as an example of what they can already do:


(h/t to Liz Barber)

The video below shows the how. It highlights four key challenges with motion capture and projection mapping in live performance.

Saturday, July 9, 2016

Prospero meets Ariel at the Imaginarium

More fascinating glimpses behind the scenes on preparations for the Royal Shakespeare Company's new production of The Tempest: "Simon Russell Beale, playing Prospero, and Mark Quartley, playing Ariel, meet for the first time, coming together with Imaginarium crew and Intel technology to explore how Ariel's avatar will work."


Thursday, July 7, 2016

Performance Capture for Stage, Film and Games

A round-up of links for further exploration of performance capture, offered with respect and gratitude to the massively talented graduating RADA actors who participated in the first-ever Shadowpox workshop – Fehinti Balogun, Natasha Cowley, Sayre Fox, Skye Hallam, Tom MartinPolly MischAbraham PopoolaMaisie Robinson and Jamael Westman.




Tuesday, June 14, 2016

Science + Fiction + Theatre

[This post was originally published in March 2015, but I'm reposting it because it's good background for a workshop I'll be doing next week. As collaborator Simon Eves of PLASTIK pointed out recently, there's still not enough sci-fi onstage...]

At a recent conference on public health history at the University of Toronto, I had some intriguing conversations about crossovers between science and art/entertainment, particularly how science fiction can welcome audiences deep into issues in public health.

It brought back a chat I had last year with Conall Watson of the Department of Infectious Disease Epidemiology at the London School of Hygiene and Tropical Medicine.

He described his work on a promenade theatre event called Deadinburgh, set during a zombie outbreak besieging the Scottish capital.

http://www.lastheatre.com/2013/02/27/the-enlightenment-cafe-deadinburgh-2/
His team ran the public health advisory cell, "tasked with guiding the audience through different approaches to controlling the zombie epidemic; giving them insight into the usually back-of-house practices of the public health authorities.

We also had input into the epidemiological parameters and narrative of the overall show."

In a paper titled "Deadinburgh: zombie epidemics, citizen power and public health", he and his colleagues Kate Harvey and Nigel Field of University College London, describe the scenario:

"An unknown pathogen was ravaging Scotland’s capital in April 2013, turning unlucky infected souls into bloodthirsty, ambling beasts. The city was under military lock-down and scientists were working around the clock to identify the pathogen and develop means of control.

"Each night, 250 uninfected citizens reached the safe zone at a former veterinary college, taking democratic responsibility for the public health and military response.

"Whether immersive theatre and simulated situations can get people to engage with public health on a larger scale and help build trust and empathy with the way that science is used to inform public sector decision-making remains to be seen. What we do know is that people like science; people like zombies; and the two combined can help us to reflect on our own practice as public health professionals."

In the video below, Kate Harvey says, "Bringing in something from popular culture helps to appeal to a wider audience.... Public health has both art and science at its core. Public health is the art and science of promoting health and preventing disease and prolonging life.... But maybe what we haven't done so much of is using it as a means of communication, and actually putting some of the science back into art as well."

Click here to watch a video of Conall Watson and Kate Harvey discussing "Deadinburgh - the science of zombies" – a London School of Hygiene and Tropical Medicine podcast.




Tuesday, April 12, 2016

So long, green screen

As I work with a Kinect, a projector and a digital video camera to drive and record real-time interactive projected effects, I'm already fantasizing about hacking the Lytro Cinema system together with one o' these.

Come on, it's only a $125,000 rental...



(H/t Jos Humphrey)

The Invisible Pantry


Wednesday, February 17, 2016

Live-Animated Simpsons

This just in from the Hollywood Reporter: the May 15th episode of The Simpsons will feature a three-minute segment with Homer Simpson taking questions... live on air. In a genre of animation that normally takes at least six months to produce, this is a 180
Showrunner Al Jean told THR that the series... will use a motion capture technology in which Homer's voice and motions will be depicted in an animated scene talking about things he "could only be saying live on that day."
"HOMЯ", Simpsons episode 9, Season 12
Apparently the segment will look just like the rest of the episode, which means they'll probably be using a 3D model, but toonshading it to flatten the look, which is what we did on Faster than Night and The Augmentalist:

Sunday, February 7, 2016

What is twenty-first-century magic?

More from the team behind the Royal Shakespeare Company's performance-capture Tempest:
"What's twenty-first-century magic? Well, I guess it comes in some kind of digital form these days. We've started to see a lot more in film, in the world of movies, but we've never really explored that in the theatre context.... It's the creative process that drives the technology, and not the other way around. So it started to be a match made in heaven, from that point of view." 
– Stephen Brimson Lewis, Director of Design, RSC




"With The Tempest we're really trying to redefine theatre, in some respects, and find a way to bring in new digital technology and really leverage it to make the story deeper, to find new ways to connect with a character, and maybe a different audience... an entirely new generation."
– Tawny Schlieski, Research Scientist, Intel

"The most exciting thing about what we're doing at the moment is enabling an actor to have a real connection with an avatar. It's the sort of thing that I do on a daily basis in the studio for films and video games. In this instance, we're doing something that's going to be a live theatrical experience. What you see is what you get." 
– Ben Lumsden, Head of Studio, The Imaginarium

"The play demands a spectacle. There's a masque in the middle of the play, an insubstantial pageant, which fades into nothing, but it creates wonder. I want to let the guys at Intel know a bit about what that tradition was, just how elaborate those masques were, and how they were pushing the envelope." 
– Gregory Doran, Artistic Director, RSC 

Thursday, January 14, 2016

Capturing Coldplay

The making of Coldplay and The Imaginarium's Adventure of a Lifetime video – one of the best behind-the-scenes on performance capture I've ever seen, and definitely the most fun!



And here's the original video: Coldplay – Adventure of a Lifetime.

(H/t to Neil Richards)

Wednesday, January 13, 2016

So this goes on the Christmas list...

New prototype high-speed projection mapping system – at CES 2016, Panasonic will unveil its projector technology for high-speed and synchronized mapping on unmarked target objects:



"Prototype." I hope that doesn't mean Christmas 2018...

Monday, January 11, 2016

"Stratford upon Avatar"

(Sorry, couldn't resist nicking the Daily Telegraph's headline.)

Very exciting announcement today by the RSC:
"Today the Royal Shakespeare Company announced a new production of William Shakespeare’s late play The Tempest, produced in collaboration with Intel and in association with leading performance capture company, The Imaginarium Studios. The companies will combine their passion for storytelling and innovation to create a truly revolutionary production as part of the RSC’s winter season 2016.

Directed by RSC Artistic Director, Gregory Doran, with RSC Associate Artist, Simon Russell Beale, as Prospero, and designed by Stephen Brimson Lewis, this partnership will see the RSC’s skills at theatre-making come together with Intel’s digital innovation and the expertise of The Imaginarium in pushing technical boundaries to create a truly innovative production for a new generation to mark Shakespeare’s 400th anniversary.

For the very first time, performance capture technology will be used to render an animated character - Ariel the sprite - live on the Royal Shakespeare Theatre stage.

The technology works by capturing an actor’s facial expressions as well as their movements, which ensures that an actor’s full performance is translated into the animated character. It has most famously been used in films and gaming, but together the RSC, Intel and The Imaginarium have undertaken more than a year of research to bring digital avatars to life on stage in real-time, interacting with live actors."
Intel expands:
"Today the Royal Shakespeare Company announced a new production of William Shakespeare's play The Tempest, produced in collaboration with Intel and in association with The Imaginarium Studios. The companies will collaborate to create a revolutionary production as part of RSC's winter season 2016. For the first time, performance capture technology will be used to render an animated character live on the Royal Shakespeare Theatre stage. Standard Intel technology is used to manage massive data processing required for live digital content projection - from Intel Xeon to Intel Core i7 processors."
I heard some tantalizing murmurs about this plan last summer, and can't wait to see what they and The Imaginarium cook up together!


The Telegraph's report is the most extensive so far: