radish lipsync 4 REDkit

radish lipsync 4 REDkit is an updated version of the lipsync animation generator feature from the radish community modding tools. The package is streamlined and adapted for usage in REDkit projects and supports two main usecases:

• []generation of lip movement animations synced to prerecorded audio

  []generation of lip movement animations based on input text only if audio is not available

  Multilingual preschool education in 13th century (click on image for video)

  Text to Lip Animation, also 13th century (click on image for video)

  Note on quality: given the limited resources (free time, zero underlings) this is most probably about as good as I can get it without investing significantly more time.

  Main Features

  • []generate lip animations from audio

    []generate lip animations from text

    []support for multiple languages (de, en, fr, pl, ru) ()

    []support for different animation profiles for different actors ()

    []generated animations can be used in REDkit scene editor and/or loaded with the REDkit blender plugin

    [](optionally) automatically pack as w3speech file

  () English is included in the download package and Polish, French, German and Russian language support is available as optional drop-in add-on package.

  (*) Lipsync profiles are based on the poseweights from REDkit patch3 blender plugin and adjust animations for different head meshes.

  Compared to REDkit lipsync solution

  • []generally higher quality lip animations ()

    []easier voiceover management (no blender required, no manual audio downsampling required)

    []bulk generation of animations

    []heuristic quality assessment: detection and visual cues for common problems (unusually long/short phoneme blocks, timing gaps in words, etc.)

    []easy to interactively adjust specific timings in GUI

    []easy to use multi language support for supported languages

    []easy assigning of a specific animation generation profile to actors in GUI

  () It's not perfect and there are cases where timings can be screwed up. However it is very easy to adjust the timings in the GUI. My general advice is to check quality for voiceover lines used in close-up shots of scenes and if necessary tweak the timings for these lines.

  The package contains a HOWTO text file with more details and more options but the general gist is:

  []unpack tools package (radish.lipsync. 4. REDkit zip)

  []unpack project template package, read HOWTO and adjust set