Manipulation Outside of VR
Manipulation in VR
Cropping and Slabbing
This tutorial covers basics of using ChimeraX virtual reality to view DICOM medical imaging data. It assumes you have the necessary equipment for virtual reality already installed and set up for the room. ChimeraX works with virtual reality (VR) systems supported by SteamVR, such as HTC Vive, Oculus Rift and Samsung Odyssey headsets. Descriptions below are for Vive hand controllers, but other hand controllers have buttons that are generally similar in position and function.
Other than the manipulation in VR, however, the tutorial can be done without VR just using your computer and mouse (or trackpad) instead of a headset and hand controllers.
See a video of this tutorial (Feb 2019). See also: ChimeraX DICOM Reference, ChimeraX for Medical Image Analysis
4-24533.zip – one chest scan of many in the RIDER Lung CT collection from the Cancer Imaging Archive:
Clark K, Vendt B, Smith K, et al. The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository. Journal of Digital Imaging. 2013; 26(6): 1045-1057.
The data have been anonymized and are freely available. This sample data folder should be downloaded and unzipped. Although this tutorial does not include clinical analysis, a significant feature of the scan is a tumor in the right lung.
Start ChimeraX by clicking its icon. After the ChimeraX window appears, open the sample data:
Menu: File... Open DICOM Folder (dialog shows Format: DICOM image), browse to location, select directory, click Choose
Just one Z-plane is shown initially. The Volume Viewer tool shows a histogram of the data with plane as the chosen display style. Below the histogram is a slider for viewing different planes along Z. Try dragging the slider. The tumor in the right lung can be seen at approximately Z-planes 146-185.
The small squares connected by a yellow line on the histogram are thresholds, the control points for mapping values to colors and intensities. They can be moved by dragging in the Volume Viewer histogram, and added and deleted using the context menu (right-click or Ctrl-click depending on platform).
Try moving the model around:
||left mouse button|
|translate in 2D
||middle or right mouse button|
|trackpad + Alt|
||mouse scroll wheel|
|trackpad 2-finger drag (except pinch on Mac)|
Some useful commands for viewing DICOM data outside of VR:
To turn off perspective, which makes closer parts appear larger on the screen: camera ortho
To center and scale the display to fit in the graphics window: view clip false
In virtual reality, the hand controllers are used to click icons and move models. There is currently no virtual keyboard in VR for typing commands.
Show the Density Map Toolbar
of icons that act on volume models
(menu: Tools... Toolbar... Density Map Toolbar).
|click image to enlarge, or see diagram at Vive website|
The Mouse Modes for Right Button
toolbar should already be shown:
Hide sets of icons not needed for this tutorial (in menu: Tools... Toolbar, uncheck Graphics Toolbar and Molecule Display Toolbar).
|The SteamVR status window shows icons for the goggles, two controllers, and two base stations in green when they are detected as ready for use.|
Take a moment to look at the hand controllers: Vive controllers each have a trigger to “pull” with the index finger, a grip button on the side, and a larger round trackpad on the top surface that can be pressed with the thumb. Above the trackpad is a smaller menu button marked with horizontal lines, and below it is a power button and a green light indicating when the controller is turned on. Buttons on other controllers are similar.
Command: vr on
(SteamVR starts automatically, providing you had previously signed in to it from the same computer and chose to have your password remembered.)
On the screen, you can see that the icon bars have been moved to the right side of the graphics window. In the process, other tool panels may have been vertically compressed; drag to make the Volume Viewer panel tall enough to show the full height of the histogram.
Put on the headset, and try moving the model. The hand-controller positions are shown in the headset as cones, and icons on the cones indicate button assignments.
||rotate hand controller with trigger pressed||
The icon on the cone “underside” shows that the trigger is assigned to translation (and rotation), and the icon on the side shows that the grip button is assigned to center and rescale.
|translate in 3D
||move hand controller with trigger pressed|
|| move hand controllers farther apart or closer
together with both triggers pressed
– or –
move hand controller vertically with trackpad pressed
|center and rescale
(recover the view)
|click grip button|
|show, hide, move control panel||use menu button|
Continue moving the model as you wish throughout the tutorial.
It is important to avoid flickering in the VR headset, as this may cause severe and long-lasting nausea. Flickering generally indicates that rendering is too slow to keep up with head movements or other changes in the scene. Rendering planes or a cropped volume is faster than rendering the full volume. Another way to decrease computational demands is to subsample the data; clicking changes to step 2 (using half as many points in each dimension), whereas clicking returns to full resolution. Step size may increase automatically when the amount of data displayed increases, but it can still be adjusted manually.
Click the menu button to show the ChimeraX control panel in the headset. The control panel is just the set of tool windows to the right of the graphics window in the desktop view, including icon bars. In VR, it can be moved around with the menu button pressed, or hidden by just clicking the button again.
A Density Map icon can be clicked with any controller button (trigger, trackpad, or grip) to give the same result.
Clicking a Mouse Modes icon with a trigger, trackpad, or grip button assigns whichever button was clicked to the corresponding function.
An icon will pop up slightly when the cone tip is in position to click it.
Leaving triggers with their default movement modes (rotation, translation, zooming) is recommended, but up to four other functions can be assigned, to the grip and trackpad on each controller. If a trigger is accidentally reassigned, using it to click either the rotate or translate mode icon returns it to the default behavior.
A single Z-plane is displayed. To view different planes along Z:
To view planes along X, Y, or Z:
|Desktop view including the VR scene in the graphics window: orthoplanes, control panel, and cones with icons for current mode assignments.|
To view planes along X, Y, and Z at the same time:
With 3D display, cropping is useful for hiding data that may obscure or distract from a region of interest.
The 3D presets are based on definitions in the Horos program for medical image visualization and analysis. Several are available from the command line, but only “Airways II” is currently available as an icon. If you later go back to viewing planes or thin slabs, it may be helpful to restore the initial thresholds by clicking .
|VR scene during cropping.|
Similar to planes, you can move a slab along X, Y, or Z:
A 3D preset like “Airways II” sets the thresholds to specific values. They can also be adjusted continuously by hand:
Using this mode, vertical motions shift all the thresholds in parallel to higher or lower values (left ↔ right on the histogram) and horizontal motions move the thresholds symmetrically farther apart or closer together (broadening or narrowing their range of values). The threshold positions on the histogram do not update until the button is released.
If you want to return to the “Airways II” threshold settings, click . For settings better suited to planes or thin slabs, click .