Rokoko Vision - uses AI to generate the animation from video.
Notes: We set up a scene with a white background and blue carpet, so that we had a high contrast against the model (Charlie).
Charlie moved around to test the video out with a webcam on a tripod. Then we uploaded the video into Rokoko studio on our computers. It takes a while to upload the video and then to generate the animation.
The angles were slightly wrong as the boards behind were not straight. The result is not perfect but it is very clever how it works. I would need to adjust things slightly to make it better.
The file can be imported into blender with a skeleton to create an animation from, with a character that I have designed. All you need to do then is create the background scene around it.
I did some more research into Mocap because I liked the idea of using it for my project:
Motion video capture refers to the process of recording moving images and audio using a video camera or similar device. It involves capturing a series of individual still images (frames) in rapid succession and then playing them back in sequence to create the illusion of continuous motion.
In motion video capture, each frame typically consist of a grid of pixels, and the frames are captured at a specific rate, usually measured in frames per second (fps).
When these frames are played back in quick succession, the human eye perceives them as continuous motion. The quality of motion video capture is influenced by factors such as frame rate, resolution, colour depth and the capabilities of the camera or recording device.
Motion video capture has become an integral part of modern multimedia technology, enabling the creation of movies, television shows, online videos etc as well as supporting various scientific and industrial applications where visual data needs to be recorded and analysed.
Motion Capture (mocap) technology is commonly used in 3D animation to bring realistic and lifelike movements to animated characters and objects. These are some of the ways mocap can be used for animation:
Capture Real World Movements:
Mocap systems use sensors and markers to record the movements of a human actor in the real world. these sensors can be attached to key points on the actor’s body or even objects to capture their motion.
Data Collection:
As the actor performs various actions or movements, the mocap system records data about the position and orientation of the markers in 3D space. This data is typically recorded in terms of X, Y and Z coordinates and can include information about joint angles, rotation and velocity.
Data processing:
The captured mocap data is then processed to clean it up, remove noise and synchronise it. This may involve filtering the data to eliminate any unwanted movements or inaccuracies.
Mapping to 3D characters:
The processed mocap data is mapped onto 3D character models in a 3D animation software or engine. This mapping involves associating the movements of the mocap data with the corresponding joints and bones of the 3D character.
Fine Tuning:
Animators may need to fine tune the mocap data to ensure that it matches the characteristics and style of the particular character or scene. This can involve adjusting timing, exaggerating movements, or making other creative changes.
Rendering and texturing:
Once the mocap data is applied to the 3D character, the animation can be rendered and textured to create a visually appealing and realistic final result. Lighting, shading and other visual effects are also added.
Integration into the scene:
The animated 3D character can then be integrated into the larger scene or environment, which may also be created in 3D software. This includes interactions with other characters or objects.
Rendering the Final Animation:
The entire scene, with the 3D character and the mocap-driven animation, is rendered into the final video or film format. This often involves additional post production work such as compositing, sound design and editing.
Using mocap in 3d animation offers several advantages, including the ability to achieve highly realistic and natural movements, save time and effort in the animation process, and maintain consistency in complex character animations. Its particularly valuable in applications like video games, film and television production, and virtual reality, where realism and immersive experiences are essential.
Comments