PatchWorld's technology revolves around a proprietary visual programming language, designed for creating interactive audio-visual 3D experiences in VR, with support for real-time multiplayer and recording of player movements. This system enables real-time synthesis of sound, audio-reactive visuals, and spatial interfaces, revolutionizing how users engage with VR, desktop, and cloud environments.
At the heart of PatchWorld is a custom audio engine, written in C++, optimized for real-time, spatial sound synthesis. The audio engine is complemented by a 3D user interface, implemented in Unity, which allows the user to build custom audio-visual algorithms, by connecting various “blocks” and interfaces. Although the audio engine is optimized for sound, it may be used as the controlling factor in visual experiences, where the actual 3D rendering is provided by Unity. The blocks that make up the PatchWorld programming language (building tool) may be complemented by the inclusion of external assets, such as 3D meshes, images, audio samples, video recordings, and more.
User’s creations may be saved to the cloud and thus become accessible to other users. That way, we create a social platform where users can create and share 3D experiences with one another. Full experiences are known as “worlds”, whereas reusable collections of blocks are known as “devices”. A user may decide to charge “coins” for their creations, facilitating a player-driven in-game economy. Users can reuse, remix, and build upon other users creations.
In addition to sharing creations with each other, players may collaborate or “jam” in real-time using online multiplayer. From anywhere in the world, multiple players may meet in an online session, and create or play together. Multiplayer also enables events such as concerts, workshops, plays, etc.
While playing, a player has the option to record all their movements and actions in what we call a “ghost recording”. Such recordings may be played back later, trimmed, and layered, to create complex experiences.
Unity’s multi platform support means that users are able to access all of the above features, and collaborate, across devices such as Meta, Vive, and Pico headsets, as well as mac/PC, web, and mobile devices. Where permitted, PatchWorld supports creation and consumption of mixed-reality experiences, where virtual assets and interactions may be combined with the real world.
Want to print your doc? This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
Tech