Motion capture for Animation in Live2d

james

Hello!
My name is james,

i just want to know is there any way to animate a video using motion capture with live2d.

Thank you so much!

curio

I don't think you can animate a live2d model with mo-cap data. The movement range of a live2d model is pretty limited (by the original illustration), so you can't do the dynamic action that you can with traditional 3D model. If it's just facial expressions, you should be able to do it based on how you make your live2d model and link up parameters.

james

Yes, " If it's just facial expressions, you should be able to do it based on how you make your live2d model and link up parameters". ... yes, thats what i am asking. can u please tell me how to do that?
(its for animating a video, not live rendering for games)

Well, if its not motion capture, Drag function looks promising, but how to add custom drag funtions..

curio

I think you should check this out:
https://www.youtube.com/watch?v=SGxTmc80fMk

I think face tracking was done with FaceShift Studio and the parameters were linked with parameters of the Live2D model on Unity.

And this guy did something similar using Kinect and Unity:

(Second guy in the session)

james

Wow thanks, i got the idea!

james

is that possible to generate the animator file with the script,
so that we can load that file in animator and rendering the final video...

curio

I don't know. It may be possible if you can somehow transform your facial data into .mtn... but I don't think there is any easy way.

Why not just use auto lip-sync and do the manual facial expression like this one? It's probably much easier, faster, and better quality.
https://www.youtube.com/watch?v=KZVRVRH3BbM