Cohort Crowd Toolkit and Asset Library

PROJECT YEAR
2022
Technologies

WIld Capture's Cohort Crowd Kit Streamlines VFX Production Pipelines

Cohort is the new crowd creator tool for volumetric assets developed by Wild Capture. After two years of development, Cohort is now being used to create crowd assets for XR, virtual production and web activations exclusively for Wild Capture clients. All part of Wild Capture’s rapidly expanding platform for digital humans and volumetric video, Cohort is a crowd toolkit designed to multiply Wild Capture’s Digital Human Platform assets,  deployed at scale, to produce lifelike crowd behaviors. This will allow creators to save hundreds of hours deploying volumetric performances into crowd environments.

Cohort is deployable across virtual production and gaming (Unreal/Unity) software, applications, VR and web based needs. Tooling up now to be production ready, this tool is designed to give the user complete controllability of individual crowd assets for digital crowd environments. It is also designed to give the user access to crowd tool functions that change the overall performance of the crowd to fit the action required. All based on varied pre-recorded high quality assets created from many individual volumetric performances.


The Cohort Crowd Kit has various libraries of smart assets to choose from. Those assets are deployed into 3D environments and then duplicated to fill larger spaces, based on how many assets are needed or purchased. Standing, sitting, high energy, low energy.  Cheering and booing, as well as other prerecorded actions from each 3D asset gives the creator ultimate control. From there, performances can be randomized to create a variance of lifelike crowd movement to achieve outstanding result when scaled.  


Cohort manipulates all kinds of unique identifiers for each character with color changing wardrobe and hair options, and props such as drink cups, banners, sunglasses, jewelry and hats can also be added in these scenes. All without overloading the system with unnecessary processing, blueprinting, writing the structure and transcoding the asset into the system properly. Cohort handles all of this. All smart assets are set to come pre-rigged (skeletal structure for manipulation) with Wild Capture's markerless motion capture service. Creating even more control for those who require it.


With Wild Capture’s many engaging products from it's "Digital Human Platform" like Wild Capture's "Digital Fashion Toolkit", users can also add product placement with logos on shirts or hats, stadium banners or any of the aforementioned hats and cups.  


While creating a pipeline for the tool and wrapping it into Wild Capture’s unique line of volumetric and digital human technologies, Cohort utilizes AI to enhance the capabilities of smart assets to create responsive crowd creation for spatial media and XR.

Comparing to the industry standard: VFX tiling processes

Traditional Crowd Plating

Generally speaking, one of the primary needs in the digital crowd space is in new media FX. For many years, this would be referred to as cinema "plating" or "tiling" in a VFX workflow. This process usually involves hiring extras and taking up many hours in a day or utilizing the need to hire a 2nd Unit or a VFX Unit for this process. One of the main advantages of volumetric crowds over traditional background or tiled crowd VFX is that if the camera is moving, traditional 2D plating can only extend the illusion so far.

As the speed of pre-production can sometimes leave VFX artists and entities to be left with less than enough time to wrangle a unit to secure the crowd plates for camera positions that may not be precisely finalized until principle photography.

As has been with traditional illusions, the more viewing angle that the audience can see the illusion from, the wider the more sophisticated the illusion must be to accommodate the extra breadth. If the trick is not designed in such a way or if the audience is outside of that range, the faster the illusion breaks down. The same applies in modern three dimensional VFX. If the camera is moving, traditional background plates will be limited in their ability to produce a believable illusion.

Believable crowd performances for concerts, sporting events and other custom built creations.

In the world of new media, creative and technological expertise proliferates the idea that virtual production and digital humans are going to be in high demand in the coming years and the technology to that recreates human performances will saturate 3D spaces as data pipelines increase.

The new landscape gives creators unique ways to utilize smaller spaces to recreate endless virtual sets. And the use of digital humans as foreground, mid-ground and background is quickly becoming the solution for new media activations.

Cohort demonstrations provide multiple use case Solutions

Early demos to show Wild Capture's use of this technology are available. This technology has been used for Wild Capture's upcoming release of 3 music videos for virtual world events for Dallas Austin's Metaverse Experience, a collaborative effort between Wild Capture, the Creative Media Industries Institute, Dallas Austin Distribution and the joined artists Kaelyn Kastle and Jazzy the Rapper.

During preproduction, the Wild Capture team worked with the artists and their designers to handle all preproduction, wardrobe design communication and technical implementation so that a successful series of captures could be brought into the Wild Capture workflow.

Once all of the assets completed Wild Capture's optimization process, the assets were then delivered as UVol web streaming links for the artists and then dropped into an asset library.

From here, Wild Capture's automation and performance randomization implements do the heavy lifting. With crowd performances from 15-25 performances for these scenes, some of the crowd performance characters will be be recreated using different parts of the performance. This use of the technology is meant to demonstrate what is possible when large scaled crowds are possible and how randomized performances can blend seamlessly into content like this without giving the illusion away that characters are being recreated.

Asset Library for Cohort

In the new era of digital humans, we require the ability to repopulate and automate digital crowds for new media engagements. Artists, creators and more seek the ability to create high end activations, content or effects that recreate the lifelike experiences of engaging humans in real life, only recreated digitally.

As the ongoing search for the ability to create virtual crowds virtually efficient logistics and to do so has taken hold in the forefront of the collective media mind, many technical, creative, financial, industrial and otherwise new media content activators have sought the ability to provide users with lifelike performances that bridge the gap for seamless FX capabilities in all manners of content.

Breaking Boundaries in New Media & XR

Cohort is the new crowd creator tool for volumetric content developed by Wild Capture. After two years of development, it is now being used to create crowds for XR, virtual production and web activations. As another part of Wild Capture’s rapidly expanding platform for digital humans and volumetric video, Cohort is a toolkit designed to implement Wild Capture’s vast array of other digital human products, usually used for individual 3D assets. Only now these assets are deployed at scale. This will allow creators to deploy hundreds and even thousands of volumetric performances into single universal scenes. Deployable across virtual production and gaming (Unreal/Unity), App, VR and web based needs, it is a production ready application meant to solve problems in this space.

This tool is designed to give the user complete controllability of individual crowd assets for digital crowd environments. It is also designed to give the user access to crowd tool functions that change the overall “mood” and overall performance of the crowd.  All based on varied pre-recorded volumetric performances from individual crowd members. 

The user has a library of smart assets to choose from. Those assets are deployed into the 3D world and then duplicated to fill larger spaces, based on how many assets are needed or purchased. Standing, sitting, high energy, low energy.  Cheering and booing, as well as other prerecorded actions from each 3D asset gives the creator ultimate control. From there, performances can be randomized to create a variance of lifelike crowd movement to achieve outstanding result when scaled.  

Cohort manipulates all kinds of unique identifiers for each character with color changing wardrobe and hair options, and props such as drink cups, banners,sunglasses, jewelry and hats can also be added in these scenes. All without overloading the system with unnecessary processing, blueprinting, writing the structure and transcoding the asset into the system properly. Cohort handles all of this. All smart assets come pre-rigged (skeletal structure for manipulation) with Cohort’s Artificial Intelligence based markerless motion capture product. Creating even more control for those that require it.

With Wild Capture’s complete platform for digital humans and volumetric video, theirDigital Fabric Tool can also add product placement with logos on shirts,banners or any of the aforementioned hats and cups.  

 

While creating a pipeline for the tool and wrapping it into Wild Capture’s unique line of volumetric and digital human technologies, Cohort utilizes AI to enhance the capabilities of smart assets to create responsive crowd creation for spatial media and XR.

History of VFX Plating and crowds

Dating back to epic movies like "The Ten Commandments" (1956) and other epics of that era, recreating crowds has required the use of glass plates, matte paintings, inflatables, dummies and other creative options for creating the illusion.

Going into the 90's, VFX companies were starting to implement early VFX to help create crowds for movies like "Braveheart" followed by a an oncoming age digital effects in cinema in full effect by 1998.

Over the last 20 years, advancements in digital cinema have made it possible to achieve realistic crowd representations. Traditional practices involving VFX units and background actors have been able to deliver believable results within certain limitations. As camera movement becomes a more intrinsic part of cinematic language, the limitations of traditional plates become more apparent. The nature of a CG "plate" is effectively represented a two dimensional tool. And the more that the viewing angle changes, the less belieavable the image tends ot become.

As narrative storytelling in media drives camera movement in more creative ways, the reasoning for using complete 3D crowd models becomes more apparent. Whats more, the production schedule and resource for VFX media can sometimes be compacted, limiting the ability to change angles to accommodate photographic principles

New Media Activations for Virtual Crowds

As a use case for The Digital Fashion Toolkit, performer Kaelyn Kastle shot her volumetric performance as a music video for her song "Sunsets Suck" in volumetric with Wild Capture.

In one of the music video's key sequences, Kaelyn Kastle is seen dancing on the shoreline of a beach. She is performing in a bikini as the virtual cameras whiz by. Wild Capture's Digital Fashion Toolkit is used in this sequence to add a light cloth shaw that drifts in the virtual breeze as she performs.

Wardrobe choices for volumetric performances are critical in the process when it comes to capturing performance models. Costume Designer for Kaelyn Kastle, "Elijah the Creative" worked with Wild Capture to design wardrobe choices that made for excellent representation in the volume. "Having wardrobe that would flow nicely as the performer moves but not so much that it obstruct to much of itself was the key to creating something that engages the audience. Most have not seen something like this before." says Elijah Brown (Elijah Thee Creator).

This project page showing the use of Wild Capture's Digital Fashion Toolkit for the Kaelyn Kastle music video can be found here.

One of the things most professionals look for when trying to judge the overall quality of a volumetric performance capture (digital human) is the overall level of details in the fabric. How well you can tell the difference in contrast between different sides of the performer or the fabric? Is the acquisition keen to not lose contour or contrast? Is there enough 3D specularity in how the model is photographed? These questions and more can identify some of the ways better image quality and believability can achieved. More on this can be found on the Digital Human Blog here.

Early Proving Grounds for Volumetric Crowds

The Digital Fashion Toolkit and many other offerings that Wild Capture is presenting. Our Cohort Crowd Kit, our Digital Fashion Toolkit and our UVol web streaming technologies all get to take center stage as we go forward into the next steps in this virtual space the idea is that we demonstrate the highest quality level as this process is duplicatable and designed for high end creators and activations.

 

You can see the demonstrations, press and more on this case study, the UVol web streaming volumetric player and other applied digital human technologies at WildCapture.io. For more information on this, please reach out to Wild Capture through the contact page provided. Connect on social media.

https://www.wildcapture.io/case-studies/uvol

Original Publishing Date February 12th 2022

Related Post

Wild Capture's Digital Fashion Toolkit

Wild Capture's Digital Fashion Toolkit

Wild Capture's Digital Human Platform

Wild Capture's Digital Human Platform

Universal Volumetric Web Streaming Player

Universal Volumetric (UVol)

Cohort Crowd Toolkit

Cohort Crowd Toolkit and Asset Library