AvatarPilot: Decoupling one-to-one motions from their semantics with weighted interpolations

Physical restrictions of the real spaces where users are situated present challenges to remote XR and spatial computing interactions using avatars. Users may not have space in their physical environment to duplicate the physical set-up of their collaborators. Still, if avatars are relocated, one-to-one motions may no longer preserve meaning. We propose a solution: using weighted interpolations to guarantee that everybody looks or points at the same objects locally and remotely. At the same time, this preserves the meaning of gestures and postures that are not object-directed (i.e., close to the body). We extend this work to locomotion and direct interactions in near space, such as grabbing objects, exploring the limits of our social and scene understanding, and generating more flexible uses for Inverse Kinematics (IK). We discuss limitations and applications and open-source the AvatarPilot for general use.