SG Com 5.3 is available now
This release gives your characters a new sense of attention with a rig-agnostic inverse-kinematic Lookat system; adds speech Beat Detection to time your gestures; and introduces a lightweight Player-only library. It also brings much improved animation quality through a fully revamped nonverbal muscle model, improved systems for blinks and microdarts, and improved lip sync.
Lookat
Characters can now track a point in 3D space, whether that's the camera for eye contact or an object in the scene. Lookat moves the eyeballs, head and torso muscles. Because it works through SG Com's dynamic muscles, it responds immediately when the target moves.
Most lookat systems solve an inverse kinematics problem over a chain of bones, so they tend to be tied to a specific rig. SG Com's lookat solves for muscle activations instead of joint rotations, so it works with any rig.
How it works
Lookat works alongside nonverbal expressions. Each expression has its own gaze direction, and SG Com compensates to hit the target with minimal change to the expression itself.
An activation setting from 0 to 1 controls how often the character looks at the target. At 0 it ignores the target, at 1 it's always fixated, and values in between give a natural mix of looking and looking away.
You can update the lookat target every frame, so it also copes when the character's head is moved by other animation, such as an idle.
New API functions are provided to set the lookat target and activation, and to retrieve head joint names used for calibration.
Setup
Enable lookat per character with a quick configuration step in SGX Studio Maya (4.6 or later), where you select the left eye, right eye and eye root joints.
The character also needs seven muscles defined: eyeball pitch and yaw, head pitch, roll and yaw, and torso pitch and roll.
Beat Detection
A beat is a stressed syllable that stands out from the speech around it, and it's a likely moment for a rhythmic gesture. SG Com now detects these beats and reports them as they happen.
Beats are signalled through the engine status callback with a new status type, SG_COM_STATUS_SPEECH_BEAT.
Each beat comes with a normalized intensity. The average is 1.0, and most beats fall between 1.0 and 2.0.
SG Com doesn't animate the gestures themselves yet. Beat detection gives your own systems a timing framework to trigger gestures at the right moment.
Record of detected beats in SG Com
Quality Improvements
Nonverbal muscles
The dynamic muscle models behind nonverbal behavior have been completely revamped.
Bidirectional and unidirectional muscles now share the same physical modeling.
Acceleration is more tightly controlled, so muscles stick more closely to the dynamic settings in SGX. Head muscles no longer look faster in SG Com than in SGX.
Muscles now overshoot and settle instead of arcing. This is closer to how real muscles behave. It also reduces the apparent latency of expressions, because the peak comes earlier, and it fixes a quirk of arcing where intensity grew with the length of the expression.
Glitching in the muscle models has been virtually eliminated.
Overshoot and settling
Blinks
Blink predictions now match the quality of the SGX blink system. Previously, blinks were too rare during speech and overabundant during pauses.
Microdarts
Microdarts also now match the SGX system in prediction quality. They're more varied, covering the middle ground instead of just alternating between the far left and far right. They've also been refined to work harmoniously with the new lookat system.
Lip sync
The universal lip sync system now matches the quality of character-specific models. Character-specific training is no longer needed, except for highly custom muscle systems.
SG Com Player: a Player-only library
SG Com 5.3 includes a second, smaller library that leaves out the Engine and focuses solely on the Player. It's built for lightweight clients, such as WebAssembly implementations where Engine-side processing is handled in the cloud or elsewhere.
All Engine-related identifiers and functions are removed.
The SG_COM prefix becomes SG_COM_PLAYER. For example, SG_COM_Initialize() becomes SG_COM_PLAYER_Initialize().
"Player" is dropped from identifier names, since it's now in the prefix. For example, SG_COM_CreatePlayer becomes SG_COM_PLAYER_Create.
SGX Event Playback
A new function, SG_COM_ReceiveEvent() (or SG_COM_PLAYER_ReceiveEvent() in the Player API), lets you play pre-generated SGX animation through SG Com.
It inserts an SGX event file (.event) containing a pre-generated muscle sequence straight into the Player. This is an alternative to feeding in real-time packets from the Engine.
If the animation buffer isn't long enough for the new animation, it will be clipped.
There's no blending at the start and end of the inserted sequence yet, so you may see some discontinuities.
SGX events must be generated using SGX Production Tools.
If you'd like to know how SG Com can bring real-time, audio-driven facial animation to your next project, please get in touch.
