In this article, I’ll show you how to capture facial expressions using just a regular webcam—no iOS device or expensive equipment needed.
Whenever I searched for ways to do facial tracking, I was disappointed because I didn’t have an iOS device. I even considered spending over 100,000 yen on specialized equipment, I don’t have enough money, so I didn’t do it, though!
So let’s get started.
Download iFacialMocap Powered by NVIDIA Broadcast
If you’re using a Windows PC and NVIDIA Broadcast’s AR features are compatible with your setup, congratulations! You can get a facial tracking tool for under 1,000 yen.
https://www.nvidia.com/ja-jp/geforce/broadcasting/broadcast-sdk/resources
https://apps.microsoft.com/detail/9n01fgs2zk3x?hl=ja-jp&gl=US
Using the software is very simple. Once you start it, just change the input to NVidia BROASCAST. If everything is working correctly, a face should appear in a small window.
To move the shape keys of your model, click “Advanced Setting”.
Copy and paste the ObjectName, which has shape keys, BoneNames, and Armature.
If you have the multiple objects that have shape keys, read the official manual at the link below.
https://www.ifacialmocap.com/tutorial/blender
The application discriminates between lowercase and uppercase letters.
Check if those shape key names are the same.
Click the addon button to find the Blender add-on. Apply it from Preferences in Blender.
Your model has to have shape keys for ARkit. You can use an official model if you don’t have it.
Press “Connect to Blender” in iFacialMocap app. Next, press “waiting for connection” in 3D view in Blender.
Well done! Your model will be moving now!
I tried combining other software.
Luppet, OBS, Screen Recorder 4
This is the setting to connect Luppet and iFacialMocap.
If You have a high-quality face rig and Faceit addon, it’s easier to create an ARkit shape key set!
I’ll sell the avatar I used as a test avatar in Nov. The product will contain a glTF file, Blender file, VRM v0.x file, and Resonite file.
Looking forward to it!
Thank you for reading!
Blender Market Link: https://blendermarket.com/creators/fun-creation
