Netflix’s runaway animated success ‘Over The Moon’ needed an xR music video to showcase one of the films main songs. Different to the film, they wanted to feature Cathy Ang - the voice of the main character Fei Fei - as a real person, set within the animated world of the film.
FRAY Studio was asked to join the xR team to build the multiple worlds and environments needed for music video for Rocket To The Moon. We needed to recreate the worlds and locations from the film in a way that would function in real-time and allow this xR shoot to be truly beautiful.
As well as the locations from the film and singer Cathy Ang, there was also a heavy dose of xR VFX magic needed from Savages. The technical and client process was managed by Boum Creative and produced by Trailer Park and Mirada Studios.
The Time frame was short - just 3 weeks from commissioning to filming. In that time we has to re-model the film locations for real-time, texture, light and refine several XR scenes in collaboration with Savages, Boum Creative and the end client.
The logistics of working across multiple time zones were ever present: FRAY Studio being based in the UK, Boum and Savages are East Cost USA and the client was West Cost USA. We also knew we were unlikely to be able to make use of any 3D assets from the film due the nature of 3D modelling for feature animation and time frame of getting approval for the assets to be released.
We took a parallel approach to the running of this project. On one side we had to get the 3D modelling up and running fast. We were able to access high resolution frames from the film and some reference 3D geometry to inform the early stages of modelling. This process was informed by the understanding the main performance areas. Objects close the the camera were created with high polygon counts and intricately detailed texturing. Object that were in the far background could be created out of a couple 100 polygons to be able to maintain the balance of aesthetic and real-time that xR requires.
The project was to be delivered in Unreal Engine. The whole Unreal project was set up with Source Control Management, carefully maintained by Boum Creative. It meant that FRAY Studio, Savages, Boum Creative and the clients could work inside the same Unreal project throughout the process. Ensuring that all the different jigsaw pieces were fitting together throughout rather than trying to merge the whole thing together at the end. This allowed Savages to begin building and refining the project and cuing architecture from an early stage to ensure smooth running come shoot day.
As models were being finished in C4D, they were being directly brought into Unreal to begin the the xR world building, texturing and lighting as other models were being created and finished. This parallel approach ensures that there was no bottle neck in the process, and all team members are given the maximum possible time to achieve their best work.
A lot of work went into achieving a materiality to match the film, this involved throwing away everything we normally do to try and make a scene look ‘real.’ This included things like painting over photo textures for example. Or using modified car paint material as the water to give the solid but milky texture required.
Another big part of the first scene was creating a real-time dynamic weeping willow tree. Rather than buying a stock willow, we had to create it from scratch to ensure that we were only spending computing resources on the areas that could be seen clearly from the main camera positions. On it’s own the tree is very one sided and unbalanced, but when slotted into the scene it fit around all of the other elements. This approach meant that we could created a hump in the willow curtain for Cathy Ang to perform in front of, trim individual branches away that felt wrong from certain camera positions and to be able to dynamically use wind on cue to make the tree come to life.
With a short timeline, the clients were kept engaged throughout. 2020 proved how much can be achieved remotely. Boum Creative managed worked with the creative team and the clients throughout the process to maintain the balance between creative ambition and reality within the time frame. With all teams locked on the same goal, we feel like Rocket To The Moon showcases beautifully what xR can do.
Creative Team
Performer: Cathy Ang
xR VFX: Savages
xR World creation/sceneography: FRAY Studio
Director: Chris Riehl
Director of Photography: Pete Konczal
Producer: Trailer Park and Mirada Studios
Edit: Mirada Studios
xR Producer: Boum Creative