This Slicer extension is in active development. The API may change from version to version without notice.
- Build the extension against the newly built Slicer.
- To start Slicer from a build tree and ensure the extension is properly loaded, consider running the
SlicerWithVirtualReality
launcher. For more details, see here.
The mapping process consists of two main steps:
-
Parsing the
vtk_open<vr|xr>_actions.json
action manifest file to link controller-specific interaction paths with generic event paths. This file references controller-specific binding files, usually namedvtk_open<vr|xr>_binding_<vendor_name>.json
, where each controller interaction path is associated with a VTK-specific event path. -
Assigning a VTK event path to either a VTK event or a
std::function
. This association of a VTK event path involving a single controller with a VTK event is carried out invtkOpen<VR|XR>InteractorStyle::SetupActions()
.
The controller interaction paths are specific to each backend:
- For OpenVR: Refer to the List of common controller types and the SteamVR Input Guide.
- For OpenXR: Refer to the Reserved Paths and the Interaction Profile Paths.
As of Slicer@c7fe8657c, the provided vtk_open<vr|xr>_actions.json
and vtk_open<vr|xr>_binding_<vendor_name>.json
files in the vtkRenderingOpenVR
and vtkRenderingOpenXR
VTK modules are as follow:
OpenVR | OpenXR | |
---|---|---|
Action manifest | url | url |
- HP Motion Controller | url | url |
- HTC Vive Controller | url | url |
- Microsoft Hand Interaction | url | |
- Oculus Touch | url | url |
- Valve Knuckles | url | url |
- Khronos Simple Controller1 | url |
These files serve as essential references for mapping controller actions to VTK events.
The association of VTK event paths to VTK events hardcoded in each VTK modules is as follow:
- For OpenVR, refer to vtkOpenVRInteractorStyle::SetupActions()
- For OpenXR, refer to vtkOpenXRInteractorStyle::SetupActions()
Recognition of complex gesture events commences when the two controller buttons mapped to the ComplexGesture action are pressed.
The SlicerVirtualReality implements its own heuristic by specializing the HandleComplexGestureEvents()
and RecognizeComplexGesture()
in the vtkVirtualRealityComplexGestureRecognizer class.
Limitations:
-
The selected controller buttons are exclusively mapped to the ComplexGesture action and cannot be associated with a regular action.
-
To workaround an OpenVR specific limitation, each button expected to be involved in the complex gesture needs to be respectively associated with
/actions/vtk/in/ComplexGestureAction
and/actions/vtk/in/ComplexGestureAction_Event2
.
Activate virtual reality view:
import logging
import slicer
def isXRBackendInitialized():
"""Determine if XR backend has been initialized."""
vrLogic = slicer.modules.virtualreality.logic()
return vrLogic.GetVirtualRealityActive() if vrLogic else False
def vrCamera():
# Get VR module widget
if not isXRBackendInitialized():
return None
# Get VR camera
vrViewWidget = slicer.modules.virtualreality.viewWidget()
if vrViewWidget is None:
return None
rendererCollection = vrViewWidget.renderWindow().GetRenderers()
if rendererCollection.GetNumberOfItems() < 1:
logging.error('Unable to access VR renderers')
return None
return rendererCollection.GetItemAsObject(0).GetActiveCamera()
assert isXRBackendInitialized() is False
assert vrCamera() is None
vrLogic = slicer.modules.virtualreality.logic()
vrLogic.SetVirtualRealityActive(True)
assert isXRBackendInitialized() is True
assert vrCamera() is not None
Set virtual reality view background color to black:
color = [0,0,0]
vrView=getNode('VirtualRealityView')
vrView.SetBackgroundColor(color)
vrView.SetBackgroundColor2(color)
Set whether a node can be selected/grabbed/moved:
nodeLocked.SetSelectable(0)
nodeMovable.SetSelectable(1)