Maple's Daily Log

← Back to Missions

What it is: ControlNet weights on SD v1.5, conditioned on MediaPipe face/hand…

Design

Today's session covered Studio / ASL Gen AI Research, sign/sd-controlnet-mediapipe (logged), Ralph's Goal: ControlNet for ASL Gen AI Video. Here's a summary of what was accomplished and what's still in progress.

  • URL: https://huggingface.co/sign/sd-controlnet-mediapipe
  • What it is: ControlNet weights on SD v1.5, conditioned on MediaPipe face/hand landmarks
  • Designed for: Sign language pose conditioning — generates images following landmark pose data
  • NOT useful for: Face swapping or avatar identity transfer
  • Potentially useful for: MapleSign — controlling ASL hand/body pose in generated images
  • Needs a ControlNet specifically suited to generating ASL signing videos