OpenSeeFace GD Edition

A Godot renderer for OpenSeeFace. Special thanks to the V-Sekai team for their help with .vrm importing.

The receiver is located in utils/OpenSeeGD.gd and handles receiving OpenSeeFace packets and wrapping that information to be Godot-compatible. screens/ModelDisplayScreen handles the mapping and displaying of models. Heavily based on OpenSeeFaceSample.

All models should work as long as they are in .glb or .vrm format. .gltf have not been tested but might work. .tscn files created with Godot should all import correctly as well.

Quickstart

1. Download the latest release
2. Unzip all files into a directory (do not move any of the files)
3. Run the .exe
4. Start the facetracker from within the application

Controls

Enter or Space: Reset face tracking offsets

Control + Left Click: Spin the model

Control + Right Click: Move the model

Control + Scroll: Zoom the model in or out

Control + Middle Click: Reset the model's transform

Tab: Toggle the UI

Building from source

1. Download Godot 3.4
2. Clone this project and load it in the editor
3. Clone the OpenSeeFace face tracker
4. Run the facetracker via Python or via the binary if on Windows
5. Run the project

Discussion

A Discord server is available here if you need help, like to contribute, or just want to chat.

Known bugs

• after stopping the facetracker, if you don't wait long enough for the facetracker listener to stop, the program can crash on close. Probably, this is hard to reproduce
• the facetracker is not automatically closed when the program crashes

Work notes

• basic OpenSeeFace visualization
• map data to a rigged, non-vrm model
• make rigged, non-vrm models lean based on translation
• load vrm models
• map data to a vrm model
• support Live2D-style sprites
• make the raw OpenSeeFace visualization loadable
• create relay server so you can pull in other people's tracking data alongside your own (display 2+ models at once)