id	summary	reporter	owner	description	type	status	priority	milestone	component	version	resolution	keywords	cc	blockedby	blocking	notify_on_close	platform	project
8375	Quest 2 VR viewer of ChimeraX GLTF scenes	Tom Goddard	Tom Goddard	"Allow viewing ChimeraX exported scenes in GLTF format on standalone VR headsets such as Quest 2.

This is similar to #8129 which proposes that the standalone headset also send hand-controller interaction back to a running ChimeraX which sends back updated scenes.  The current ticket is simpler in that it does not offer VR control of the ChimeraX user interface.

{{{
From: Tom Goddard 
Subject: ChimeraX VR on Quest 2 headsets
Date: January 24, 2023 at 6:44:35 PM PST
To: ""Cruz, Phil (NIH/NIAID) [C]"" , ""McCarthy, Meghan (NIH/NIAID) [C]"" 
Cc: Chimera Staff 

Hi Phil, Meghan,

  I was dreaming today about where to go with ChimeraX virtual reality.  The Meta Quest 2 standalone headset it said to have 90% of the VR market share.  Here are some thoughts about how we could make ChimeraX VR much more widely used with Quest 2 headsets.

  The basic idea is to have ChimeraX export scenes, probably as GLTF, that the Quest 2 renders standalone.  In other words the GLTF scene is transferred to the Quest 2 wirelessly possibly directly from ChimeraX or through a file sharing service like Google Drive or Dropbox.  Then the Quest 2 does everything to render it in VR.  Here are advantages.

A1) You don't need a VR capable computer.  Probably only 1% of ChimeraX users have such a computer.  You can use a laptop or any computer you usually use for ChimeraX.
A2) Quest 2 is by far the most widely used so more researchers already have it compared to old Windows PC tethered headsets.
A3) A researcher only needs someone's Quest 2 to try it, and it is cheap to purchase if the find it useful ($400-$500).
A4) No wires, no base stations, so easier to use at the researcher's desk.
A5) The VR user interface will be much less complex, since it is not going to control ChimeraX.
A6) Scenes will be copied to the Quest 2 and so ChimeraX is not needed at all to show others science models.
A7) Multi-person sessions are more likely to happen because more researchers have equipment.
A8) ChimeraX use will be normal desktop mode, no switching between VR mode and desktop mode.
A9) It is portable to new physical locations (e.g. between home and work), rather than fixed to a dedicated computer and room.

And here are disadvantages.

D1) The standalone Quest 2 graphics is not as powerful as PC graphics, maybe 5x slower, scenes must have less than 1 million triangles.
D2) Updating a model requires taking off headset to use ChimeraX.  The VR is more like a special display.  Can show/hide parts of models in headset or switch between models previously loaded on headset, so some ability to change what is shown is possible without ChimeraX.
D3) ChimeraX use requires some skill to simplify scenes.  Can make a ChimeraX tool to reduce atom/bond/ribbon/map quality to stay within VR triangle limit.
D4) Requires a new Quest 2 app, probably written in Unity.  Many have tried similar things, e.g. Enduvo, Arkio, AltSpace, ....
D5) Might want some special rendering capabilities on headset, e.g. volumetric rendering or ambient shadows, adds complexity to Quest app.
D6) Dynamics will be inefficient if every time point needs a new scene.
D7) Multi-person meetings would ideally have built-in audio.  This is difficult and might require a commercial solution.  The more cumbersome simultaneous Zoom meeting from a laptop might not be able to use the VR headset microphone and headphones.

  Overall would be much more usable for researchers and could lead to much wider use.  This design largely separates ChimeraX from the VR rendering.  It is easier to learn and setup.  Handling the GLTF in Unity is done very well by glTFast, a free package (https://github.com/atteneder/glTFast) that appears to have excellent support.  It would be necessary to keep the Quest 2 app as simple as possible to be able to implement with our limited resources.

  I haven't put much work into our ChimeraX VR the past year.  Partly that is because I focussed on machine learning structure prediction.  But another reason is that the tethered PC VR technology seems to be in decline, replaced by Quest 2 and similar standalone competitor Pico 4 and more expensive standalone Vive Focus 3 ($1300) and Vive XR Elite ($1100).  The only notable new tethered VR headset in the works seems to be for PlayStation 5 dedicated to video gaming.  So we need to figure out how to utilize the standalone technology.

	Tom
}}}"	enhancement	assigned	moderate		VR								all	ChimeraX