Picture the scene in a Las Vegas casino – a young Elvis Presley performs on stage before a room full of fans, tweeting their excitement or recording on their camera phones as he recites his greatest hits. He can sing, dance, laugh and joke with his audience and as the show progresses, so too does Presley’s appearance: in a single evening, he changes from the baby-faced 21 year-old of the early 1950s to the Comeback King, dressed head to toe in black leather. In a room nearby, burlesque performer Betty Paige is starring in her own stage show, while in New York, Marilyn Monroe performs the story of her rise to fame on Broadway.
Just ten years ago, this might have seemed an unlikely feat: creating realistic projections and moving CG likenesses of real people was near impossible, and far too complex and time consuming to be a worthwhile endeavour. But since 2012, we have seen Tupac perform posthumously at the Coachella music festival, Michael Jackson at the US Billboard Awards and in a Cirque du Soleil show, and a digitally recreated Audrey Hepburn star in a Galaxy ad. And in the next two years, it seems a range of late, great celebrities will be returning to both stage and screen.
Pulse Evolution is a Los Angeles-based company founded in 2013 by John Textor, the former CEO of VFX studio Digital Domain. The company specialises in creating CG humans and since working on Jackson’s Billboard performance, has partnered with his estate, and the estates of Elvis Presley and Marilyn Monroe, to create digital likenesses of all three stars.
An Elvis performance is expected to launch in Las Vegas in spring next year, with a Broadway show starring Marilyn due to follow in 2017. Using a mix of augmented reality, live projection and digital animation in venues with custom-built sets, Textor is confident they will deliver a convincing, visceral experience for fans.
“The Michael Jackson projection was very limited, as we only had around 22 weeks to prepare and the stage was already set [like the Tupac performance, it used a Pepper’s Ghost projection technique, created for Pulse by London company Musion]. With Marilyn and Elvis, we can have Elvis walking down the aisles, Marilyn’s face shown in close-up on big screens by the stage, and they’ll be able to interact with audiences and respond to their environment [using real time projection and animation techniques],” he adds. Digital likenesses are still in development, but a Michael Jackson we were shown under embargo is startlingly lifelike, already capable of blinking, smiling and turning his head, though he’s still awaiting a full head of hair.
CMG Worldwide, an Indiana company which manages the rights of over 300 deceased celebrities from Ella Fitzgerald to Buddy Holly, is also planning to launch live events using digital projections of late icons. The Betty Paige show is due to open at the end of this year, while projects and endorsement deals involving likenesses of James Dean and Amelia Earhart are also being discussed.
With tickets for Jackson’s Cirque du Soleil show selling for up to $200, staging live events with late performers is a lucrative business – and audiences would doubtless pay vast sums of money to see acts like Elvis, Marilyn or the Beatles performing as they would have at the peak of their career.
Making them is still a costly business: Textor says Pulse will be working “for several months” on its digital likenesses but as he points out, “if Elvis is getting paid to do two shows a night, seven days a week in Vegas, London, New York and Seoul, that’s a lot of money in a lifetime,” – especially if that lifetime is eternal.
In advertising, too, brands have long been keen to exploit our enduring fascination with celebrities past: James Dean has reportedly made far more money through endorsement deals since his death than he ever did as an actor while alive, and in the past 20 years, Monroe has appeared as the face of various cosmetics and fashion brands, from Dior to Chanel and currently, Max Factor. Until recently, this was mostly done using old stills or film footage but in 2014, Framestore used VFX and body doubles to create a convincing likeness of Hepburn in an AMV BBDO ad for Galaxy.
Shot in the Italian Riviera, it featured the actress travelling on a bus before spotting a handsome driver and jumping in the back of his car to eat a bar of chocolate. The film used a combination of clever lighting and two doubles (one with a similar facial bone structure to Hepburn and another with a similar frame), but her face was built entirely using CGI, a feat Framestore claims had never been achieved before in close-up.
Creating photoreal humans is still the biggest challenge in VFX: in the past, it was impossible to create CG characters indistinguishable from real actors due to the Uncanny Valley effect (digital avatars would lack the subtle imperfections found in human skin, hair or eyes, which would be immediately detected by our subconscious, spoiling the effect). Now, though, sophisticated scanning systems make it possible to capture accurate 3D representations of an actor’s living form.
“With hi-res scans of an actor’s face, we can capture their exact skin tone, the shape of every pore on their face, even map blood flow rates underneath the skin to determine how long it would take them to turn red,” explains Darren Hendler, a DFX supervisor who recently created pixies based on human faces for Disney movie Maleficent. “We can easily create a static head that is so realistic, if you placed it side by side [with a real image], the actor probably wouldn’t know the difference.”
The tricky part, of course, is animating them. Creating a walking, talking CG human that looks convincing in a close-up for a film or advert is much more challenging than for a live event, where lighting, staging and set design can provide distractions – but it is possible, says Hendler.
“It’s very complicated and time consuming, it can take up to two years to create a moving, talking head that looks good up close, but as the technology progresses, we are developing new techniques that are able to bring down that time frame. It’s getting quicker and quicker, and that’s where we’re focusing a lot of our research [at Digital Domain] now,” he adds.
Creating likenesses of deceased celebrities who haven’t left behind any digital data, on the other hand, usually involves piecing together hundreds of thousands of still images and video footage, as well as the use of body doubles that can provide a ‘canvas’ from which to work, says Hendler. It’s a laborious process and as Textor points out, even photographs aren’t always accurate. “Most celebrities are filmed in make-up, which can create illusions – we wish we had a clean, desaturated photo of Michael Jackson to work with,” he adds.
Today, however, leading actors and musicians are increasingly looking to preserve themselves digitally – most already have their faces and bodies scanned when starring in a major feature film, which studios often use to create digital doubles for scenes that would be dangerous or difficult to shoot, and Hendler says Digital Domain has been involved in conversations with a number of celebrities about creating a digital legacy – “essentially, a package of material that can be used to make a digital self should anything happen to them,” he adds.
This data, coupled with voice recordings and real takes of an actor’s various expressions, movements and emotions, would give studios enough information to create a digital version of an actor capable of starring not just in a 30 second ad, but a feature length film. “Someone like George Clooney could sell his image rights to movies he is going to be in once he has died today,” claims Framestore co-founder, Mike McGee.
“Imagine if he had taken a body and facial scan every few years since he began acting – a film like Boyhood could potentially be made starring George as a totally accurate representation of the person he was at those different stages of his life … that’s why today in Hollywood, young actors are rumoured to be having their bodies scanned at regular intervals and stored on computer. If they take the time to record physical performances and voice recordings, who knows what might be possible in the future.”
Not every actor’s image remains a valuable commodity after their death, but for the most successful, protecting and preparing for a digital afterlife may be a wise (or at least, very profitable) decision. With celebrities now capable of recording an album or touring even after their death, lawyer and CMG Worldwide founder Mark Roesler says stars are drawing up increasingly comprehensive plans for how and where their image or likeness may be used, and who should benefit from it, should anything happen to them.
“A lot has changed since I started the business [in 1981], both in the recognition of the value of intellectual property rights and the myriad opportunities available for deceased personalities to earn revenue streams – sometimes, very significant amounts,” says Roesler.
“Typically, stars aren’t worth more deceased than they are alive, because you lose all the personal services associated with that personality, but now it’s possible for them to continue to live on, and that’s adding a whole new dimension [to our work] … we represent a lot of living celebrities … and usually, their intangible assets [ie image rights] are worth more than their cars, houses or the money in their bank accounts,” he adds.
While performers today are ensuring a say in how their digital double might be put to work post mortem, however, Presley, Monroe and Hepburn have had no such luxury. In their time, it would have seemed like the stuff of science fiction, and it’s highly unlikely they ever discussed the possibility of advertising a chocolate bar or performing at a casino from beyond the grave. So is it morally wrong to use their likeness in this way?
Roesler doesn’t think so, adding that whoever owns the rights to a star’s image, whether family, spouse or lawyers, should be able to use those assets however they please. Digital likenesses and projections are no different to using a star’s signature, voice or still image, and estates have to grant full permission before they can be used in films or ad campaigns.
“People are always going to have an opinion on what a celebrity says or does or endorses, whether they are dead or alive – that’s just the nature of celebrity. If you drive a purple car, I’m entitled not to like it, but if you want one, that’s your prerogative – who am I to impose judgment when you own what you own?,” he says.
After working on the Galaxy ad, McGee says Framestore was inundated with enquiries, “from bringing back deceased singers to record new songs, to making short films with a collection of actors who had never previously performed together … when we do our job well as VFX artists, there is the inevitable assumption that it’s easy to do again,” he adds.
Textor claims he was even asked to create a digital Pope and Barack Obama after the Billboard Awards, (unsurprisingly, not by the Vatican or the White House), while Digital Domain project manager Tiffany Tetrault noted a similar level of interest after Tupac’s Coachella appearance.
“We had phone calls from everyone under the sun, even individuals asking if he could perform at weddings and proms – it will be interesting to see what happens if large corporations get ownership of deceased people, whether musicians start appearing at fundraisers and the like,” says Tetrault. “I’m sure there would be people horrified to see themselves advertising certain products,” adds Hendler.
For brands, McGee says having a virtual celebrity ambassador is “simply gold dust” – but it’s not just actors and musicians that may soon be brought back to life.
“Imagine being able to enter a virtual world in an Oculus headset, and have a chat with Audrey Hepburn – through voice recognition and artificial intelligence, she could answer any question you might want to ask about her. Expand that to any historical figure, deceased celebrity or political leader and the possibilities are endless,” he adds.
Textor agrees, and expects the impact of creating convincing digital likenesses of celebrities past will lead to investment in CG humans and lifelike projections in other industries too, from science and medicine to education, creating a future where digital people are the norm.
“Once people see how realistic we can make a digital Elvis or Marilyn look, it will create more of a conversation around what’s possible, and that’s when people will really start investing in it. You could project digital teachers into schools in hard-to-reach places, use CG humans for surgical training – we’ll probably have digital avatars of ourselves that we can use to communicate, or even just play video games. The tech exists, so it’s just a matter of time,” he adds.