Virtual Human Toolkit 2.0 Tutorial

Virtual Human Toolkit 2.0 Tutorial

Background

To fully explore the possibilities of intelligent virtual agents (IVAs), it is imperative that researchers and developers are able to rapidly create and iterate on IVA systems. This is not a trivial undertaking, given that these are large, complex systems, often lacking specific frameworks or solid standards. Furthermore, certain principles may only be understood in a narrow context and can be difficult to generalize across multiple domains, which limits the ability to re-use knowledge and assets, often resulting in the need to start new characters or systems from scratch.

To address these challenges, the University of Southern California (USC) Institute for Creative Technologies (ICT) created the Virtual Human Toolkit (VHToolkit), designed to aid researchers with the creation of IVA systems [1]. Originally released in 2009, the VHToolkit enables further research by offering a collection of IVA modules, tools, and libraries, as well as a framework and open architecture that integrates these components.

Ongoing efforts aim to modernize the VHToolkit, with the goal of providing a flexible platform for rapid prototyping and experimentation while leveraging robust commercial technologies. In this, we are guided by the following desiderata:

  1. Modularity: mix and match technologies as needed
  2. Extendability: add new technologies and expand existing ones
  3. Interoperability: use existing work and avoid vendor lock-in
  4. Multi-platform: support desktop, mobile, web, and XR
  5. Community: share results and propagate robust technologies
  6. Re-use of data and technology: leverage previous efforts
  7. Cloud and local: leverage the cloud while allowing for network-constrained and low-latency required environment

These efforts will result in the Virtual Human Toolkit 2.0, slated to be released by late 2023. For more details on the guiding principles of this effort, see [2].

Tutorial Overview

Attendees will learn about the VHToolkit 2.0 architecture and technologies, and get hands-on experience in creating a basic IVA system.

This tutorial includes the following elements:

  • Demonstration of VHToolkit 2.0 main example
  • Overview of the general architectural concepts
  • Overview of the main technologies
  • Hands-on on how to install and run the main example
  • Hands-on on how to create a new project and new content
  • Hands-on on how to set up and connect to cloud services
  • Hands-on on how to deploy to multiple hardware platforms (e.g., mobile, web)
  • Introduction on how to extend the VHToolkit 2.0 API and add new technologies

The length for this tutorial is half a day.

Target Audience

This tutorial is aimed at researchers and developers within the field of IVAs, including computer scientists, social scientists, designers, and programmers. A basic understanding of Intelligent Virtual Agents is assumed. There is no prerequisite for any specific in-depth knowledge, but a general affinity for technology is required for the hands-on portions of the tutorial, preferably including the Unity game engine, C# programming language, and source control software.

Participation Requirements

While not required, attendees are encouraged to bring their own laptop, in order to be able to follow along with the hands-on portions of this tutorial. Windows is preferred, although many features also work on macOS. A list of 3rd party software to download beforehand will be communicated shortly before the workshop. Setting up cloud services accounts may require a credit card.

References

  1. Hartholt, A., Traum, D., Marsella, S.C., Shapiro, A., Stratou, G., Leuski, A., Morency, L.P. and Gratch, J., 2013. All together now: Introducing the virtual human toolkit. In Intelligent Virtual Agents: 13th International Conference, IVA 2013, Edinburgh, UK, August 29-31, 2013. Proceedings 13 (pp. 368-381). Springer Berlin Heidelberg.
  2. Hartholt, A., Fast, E., Li, Z., Kim, K., Leeds, A. and Mozgai, S., 2022, September. Re-architecting the virtual human toolkit: towards an interoperable platform for embodied conversational agent research and development. In Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents (pp. 1-8).