Wednesday, February 08, 2012


"I wonder how long it will be before soap opera writers and movie producers are using AI [Artificial Intelligence] improvisations to create content. After all the three act structure that governs most mainstream film releases is so intricately shaped, it is almost a computer program in itself; start combining the data from successful movies, and allowing AI characters to experiment within the resulting confines and you have the sort of crazed hybrid of Hollywood formula and Marxist dialectics that can only result in hit romantic comedies."

The quote is from producer Sean Coleman who heads the team responsible for Channel 4 Education's new online drama named 'SuperMes', about a shy loner moving into a shared house with three lively characters. The series uses virtual actors and settings from popular computer game 'The Sims' (from an article about emergent drama here). 

Reflecting on this the movie 'Simone' came to mind, filmed exactly ten years ago. Remember? It told the story of a producer whose star walked off the set and left him with a big problem. His solution was a 'Sim', a digital actor who was photo-realistic to an extent not actually possible back in 2002. Today for sure there is technology almost able to deliver far more advanced verisimilitude... But prompted by what Sean Coleman had to say by thinking went off in other directions... towards a near future in which human actors have still not been replaced by avatars but where technology enables new ways of creating content.

Only a couple of years before 'Simone' came out I had been fortunate to spend a year producing serial drama using green screen tech not just for a few 'special effects' sequences but on a regular day-for-day basis. Since the dramatic action took place on an express train, moving backgrounds were essential and this posed an interesting challenge for multi-camera studio production.

The background footage had been shot from five different angles and stored as five synchronous feeds on a server. While shooting each cameraman had to pick the background feed which best corresponded to his perspective of the foreground action and by and large the end result was extremely convincing. It was also fascinating to have the task of integrating young computer geeks with veteran television crafts people to form a harmonious, efficient and productive team.

To be perfectly honest it was only the technology aspect of 'CityExpress' which can be recalled as a success... the series had many, many flaws and it was not renewed by the network for a second 40-episode season.

My thoughts will, however, return to green screen production later.

But what about the possibilities technology offers for the earliest stage of story generation?

The storytelling for episodic serial drama traditionally starts with what is commonly called the 'bible'. This outlines the long to medium term narrative arc development but importantly also sets forth the constraints in terms of the core characters and their relationships, the settings in which the dramatic action takes place and establishes the emotional tonalities of the series.

Conventionally the 'bible' is a hefty but succinctly formulated text. If I understand the implications of the 'SuperMes' project correctly, then the 'bible' can also be extended as a parallel collection of metadata, conformed to be subject to intelligent algorithms. This would enable the parameters contained in the 'bible' to become not only 'rules' binding for the subsequent work of the storyliners but would allow for autonomous AI contributions to the stories as a result of algorithmic interaction.  In this way the computer's program becomes one of the valuable creatives sitting at the table in the writers' room.

Coleman again...

"We spent a number of weeks working out the personality traits that would work best to give us the most interesting characters, the most dramatic conflicts and the most quirky entertainment. The game is very sophisticated in the grey areas that it provides... the kind of ambiguities of personality that make really fascinating characters, and we were constantly surprised by our Sims. Like the time the two boys decided to go skinny-dipping. Nothing we had done told them to do that."

Would an AI presence among the storyliners working on an episodic drama series stifle creativity, lead to predictable formulaic content creation? Some will doubtless take that stand. My feeling is that the opposite outcome is more likely... one which is richly positive and which could blow the cobwebs off traditional soap opera... with its roots in radio fiction back in the late 1930s... and make it bubble with the effervescence that today's transmedia multi-screen on-demand audience increasingly expects.

So let us assume that the robot... for in the context of emergent drama the term is justified... is there at the writers' conference table. At the end of the storylining week the outlines of five new episodes will conventionally be 'signed off' by the head writer before being passed on to the five writers of the individual shows. But a new step can intervene... the parsing of the new storylines by the computer program in order both to check for conformity with the metadata 'bible' and to add any new data which my have become relevant. My feeling is that at this stage certain expressions would have to be recognized by the program as significant, words revelatory of intent or emotion or cognition.

What influence can the robotic colleague then have for the next stage, the work of the five writers assigned as what the French so helpfully call dialoguistes who turn the bare-bones provided by the storyliners (or scénaristes) into final shooting scripts for a subsequent week of studio production? Here I imagine a second open window on the writer's screen providing metadata guidance concerning the behaviours of specific characters, to make sure that consistency with the canon of the series is safeguarded.
  • When John returns to his flat at night he always goes to get a beer from the refrigerator before switching on the lights.
  • Alice always goes back to check whether her car is locked even after she has used the remote control key.
Given that the writers of individual episodes come and go during the life of a series... whether an open-ended 'soap opera' format or a long-form telenovela... such an aid would make the job of the script editor less onerous.

Now that five scripts are ready to shoot it is almost time to move into the studio. And... if we are to further espouse advances in technology... how can it not be a green screen production environment?

However before work can start in the studio the director assigned the responsibility for a 'block' of five episodes must do his prep. The technology here discussed must surely allow the director to benefit from having an AI assistant, most likely taking the role of storyboard artist.

The same virtual sets later used in the studio would be used in the storyboarding phase, with the human cast members represented by avatars. The director's preparation work would to some extent resemble the interface of the online virtual world 'Second Life'.
  • A text-to-voice app would allow the running time of each individual scene to be be fine-tuned.
  • The metadata for each setting could proscribe certain stylistic elements which might be part of the series canon... for instance, scenes set in a 'billiard saloon' might invariably start with the potting of a coloured ball in extreme foreground before framing actors beyond (although the billiard table sequences would most probably be composited CGI elements!).
  • The final storyboard output would essentially be a pre-visualization of each episode. It would not only generate 'shot lists' for camera operators but also could include inputs from the wardrobe department and provide guidance for the props crew. This would be especially helpful in determining what 'practical pieces'... items with which the human actors will come in physical contact in the virtual sets... will need to be available for the shoot.
Fully prepped and his blocking and staging choices confirmed by the storyboard, the director can now confront the human talent in a phase which might still be called 'rehearsal', although new terminologies will certainly emerge when AI-supported content creation takes hold.

[to be continued]

1 comment:

M.Sean Coleman said...

Interesting thoughts indeed! Glad our little series and its crazy parameters inspired this line of thought for you. Thanks for taking the time to forward it on. good luck!
Sean Coleman (keep in touch on twitter: @seancoleman)