Reflector
Autonomous AI engine that edits film in continuous real time

An autonomous AI engine that arranges and edits film in continuous real time. Putting this computer program in the editor’s seat opens new doors to innovative ways of filmmaking. 

Artificial intelligence is widely used in the production and distribution of films, with algorithms “deciding” when to release them to maximize audience size. Computers running hundreds, if not thousands, of scripts know in advance whether a film is going to be a hit or a flop. Statistically, AI can predict box office success three times more accurately than humans, and that includes highly experienced producers.

Based on never used footage, shot by twice Oscar-nominated director James Longley, depicting the life and trials of Pakistani orphans, the team trained an AI to edit and assemble an artistic documentary, which raises questions about what makes a film human and how we may collaborate with deep learning algorithms to expose new truths about our world. Reflector is the next step in the integration of AI into the film industry. 

RNDR developed the Reflector installation to show a glimpse of the potential role of machines in documentary and arthouse film production, and to find common ground for dialogue among filmmakers about the future of film as an art form. The initial outcomes were presented as an interactive installation at IDFA DocLab 2019. Reflector is part of the larger research project Kaspar initiated by Piotr Winiewicz, authored by Piotr Winiewicz and Dawid Gorny and produced by Mads Damsbo, Makropol.

The installation visualises how an AI chooses the different scenes as a dynamic treemap with indications of how the different scenes scoreon certain parameters. Visitors can also influence the edit of the documentary by changing three different parameters (shot length, shot colour, shot framing) on a touch screen. These three parameters stand for the total about 30 different parameters that are used to classify the scenes.

Interactive visualisation and interface to show scene ranking based on certain parameters
Visualisation of all the scenes in the database

BACKGROUND ON THE CINEMATOGRAPHY
Piotr Winiewicz acquired the rights to use more than 60 hours of pristine material shot in Pakistan in 2010 and 2011 by James Longley. Longley intended to tell the story of a country torn by war and an everlasting domestic and international conflict, through the eyes of the young boys of a forgotten orphanage. The raw material is rich, characterized by beautiful cinematography by Longley.

He envisioned the project as a feature length film, but following American CIA agent Ryan Davis’ scandal, events took a different turn. All American citizens were sent away from Pakistan including the director, who never got to finish shooting his film. The existing material is nevertheless ideal for a short­format documentary. When looking at the raw material shot by James Longley in Pakistan we immediately see the footages potential to assemble into an important film. A film that could give us an inside look at Pakistan. We believe in the idea that if you can understand some part in detail, it makes it easier to comprehend the complexity of the whole. Longley wanted to reconstruct the internal world of the orphans at Faizul Islam as a cinematic experience, and through their perspective to make a film that would bring audiences closer to understanding Pakistan as a society ­by seeing some part of it in detail.

We share his ambition for the film and acknowledge the fact that due to economical, political reasons and significantly complex film production processes, some films and their stories remain untold. It is a chance to tell a relevant story and pay respect to the hard work of the filmmaker and at the same time reveal an innovative approach to documentary filmmaking.

Reflector timeline visualisation
Date
2019
Location
Tolhuistuin, Amsterdam
Directors
Piotr Winiewicz, Dawid Gorny,RNDR
Production
Co-production
Key collaborator
Anton Marini
Executive producer
Jakob Høgel
Cinematography
James Longley
Editing
Kaspar
Design and coding
Developer
Rasmus Ravn Frost, Esbern Torgard Kaspersen
Sound Design
Rebekka Bohse Meyer
Screening copy
Winner
2019 Film Fund DocLab Interactive Grant

Nominated
Nominated Immersive Non-Fiction competition
Nominated
IDFA Competition for Creative Use of Archive
OPENRNDR
Open source framework for creative coding
Read more
UNCODE
Interactive real-time installation exposing the mechanics of creative coding for Electric Castle Festival, 2019
Read more
No_Code: Shelter
Visual language for an exhibition at Salone del Mobile, Milano
Read more
Good Governments, Happy People
Tool to inspect the indicators that might make a good government
Read more
Open Highway
A real time light installation visualizing highway vehicles in scale 1:1
Read more
The Blue Front
Using archives to dig-up locations of air attacks on the North Sea from World War II
Read more
Load more

Studio

RNDR is a design studio for interactive media that develops ‘tools’ that are only finished by how they are used.

To achieve this, we develop processes, create structures, design visualisations, code programs, and create interactions. The end result can manifest itself across different media, ranging from interactive installations, data visualisations, generative identities, prints and everything in between – often real-time. We are triggered by how information and technology transforms networks, cultures, societies, relationships, behaviours, and interactions between people.

RNDR was founded in 2017 in The Hague, (NL). Its main members have years of experience as partners, computer scientists, designers, art directors and developers at LUST and LUSTlab.

One of our core projects, and basis for most of our projects, is OPENRNDR, an open source framework for creative coding –written in Kotlin for the JVM– with over seven years of development, that simplifies writing real-time audio-visual interactive software. OPENRNDR is fundamental for the work and software capacity of RNDR as a studio, as it allows us to realize complex interactive works. OPENRNDR was awarded the Dutch Design Award 2019

People
Jeroen Barendse (NL), partner. Design and art direction. Former partner of LUST and LUSTlab. Awarded BNO Piet Zwart Oeuvre Prize 2017
Edwin Jakobs (NL), partner. Computer scientist, creative coder and visual artist. Creator of OPENRNDR
Boyd Rotgans (NL), partner. Creative coder & interaction designer
Viola Bernacchi (IT). Design & data visualisation
Els van Dijk (NL), Office manager
Ferdinand Sorg (DE), Intern
Previously at RNDR
Jaekook Han (KR), Intern, Graduate Interactive Telecommunications Program, NYC, 2019
Gábor Kerekes (HU), Developer, 2017-2019
Amir Houieh (IR), Developer, 2017-2018
Łukasz Gula (PL), Intern, Royal Academy of Art, The Hague, 2018
Noemi Biro (RO), Intern, Willem de Kooning Academy, Rotterdam, 2018
Selection of exhibitions
Open Highway, Raum, Utrecht, 2018
Typojanchi Typography Biennale Seoul, main exhibitor (as LUST), 2017
Cartographies of Rest, interactive installation at Mile End Art Pavilion in London (as LUST), 2016
Hyperlocator, What’s next, Future Tomorrow, 38CC Delft (as LUST), 2016
Type/Dynamics exhibition, Stedelijk Museum Amsterdam (as LUST), 2014
Selection of lectures
Lecture at Britisch Library for Central Saint Martins' Form, reform, perform: futures of writing, 2019
MiXit conference, Lyon, 2019
Google Span conference, presentation and demo, Helsinki, 2018
JFuture conference, Minsk, 2018
Creative Coding Utrecht, Launch of OPENRNDR, 2018
Selection of workshops
OPENRNDR x Machine Learning workshop @ KABK during Tech Week, 2020
Two week OPENRNDR workshop at Tumo, Armenia, 2019
Workshop OPENRNDR at Artez Interaction Design, Arnhem, 2019
Processing Community Day 2019, CCU and Sensorlab, Utrecht, 2019
Royal Academy of Art (KABK), Techweek, The Hague, 2019
Utrecht School for the Arts (HKU), OPENRNDR workshop, 2018
La Scuola Open Source, 1-week workshop, Bari, Italy, 2018
Two-week workshop at TUMO foundation, Yerevan, Armenia, 2019

Contact

RNDR

Paviljoensgracht 20
2512 BP, The Hague
+31 (0)70.3635776

info@rndr.studio


INTERNSHIPS

We are mostly looking for interns that have a background in interactive design. Some experience in coding is preferred. 

interns@rndr.studio

OPENRNDR
Open source framework for creative coding that simplifies writing real-time interactive software

info@openrndr.org
openrndr.org