• this presentation focuses on Mesoscale Explorer
• the 2024 Volumes and Segmentations paper is very coding-focused whereas the 2023 one mostly just says volumes and segmentations can be shown

• large, complex 3D models of biological structures (viruses, bacteria, organelles) consisting of thousands of macromolecules and billions of atoms
  
  
• typically based on multiple sources of knowledge and data types
  
  
• presented at varying levels of detail to aid comprehension
  
  
• increasingly abundant due to advances in:
  • experimental structural biology, particularly cryo-EM and cryo-ET, and growth in the resulting datasets
  • computational integrative modeling based on heterogeneous data sources: cryo-EM/ET, light microscopy, crosslinking, mutational scanning, etc.
  • systems for multiscale model-building and depiction: cellPACK, Yasara Petworld, Mesocraft

• web application for the exploration of multiscale models built on top of Mol* (the default viewer at the RCSB PDB)
  
  
• enabled by recent advances in GPU rendering power and web browser capabilities
  
  
• mainly targeted at researchers, educators, and students in molecular and structural biology, but also meant to be accessible to the wider public
  
  
• intuitive interface with detailed documentation and guided tours through various models
  
  
• supports real-time rendering of scenes containing billions of atoms, with several optimizations inspired by cellVIEW and video games
  
  
• formats: mmCIF, binaryCIF, Yasara Petworld models (mmCIF extended to multiple distinct [sub]models with instances and markdown descriptions), manifest files (JSON-based container format, streamlined binary), Mol* sessions (scene captures, as used in guided tours)

• instancing to draw multiple copies of the same thing
  
  
• representing structures with simple sphere-based-geometries
  
  
• reducing detail in objects that are farther away
  
  
• drawing only what is in front (occlusion culling)
  
  
• drawing only what is within the viewport (frustum culling)

• a: zoomed-out view in default mode
• b: "Ultra" mode, very high level of detail (LOD), e.g., more spheres per molecule
• c: "Quality" mode (default), high LOD
• d: "Balanced" mode with sphere approximation (e.g., as flat discs), medium LOD
• e: "Performance" mode, low LOD to maximize frame rate
• (explore interactively)

• contact shadowing – tracing a path from each pixel toward the light source
• multiscale screen space ambient occlusion (SSAO) – varying the occlusion radius in multiple steps
  • a: standard SSAO
  • b: multiscale SSAO
  • (explore interactively)

• a: plane
• b: sphere
• c: cylinder
• d: infinite cone
• e: cube
  
  
  
  
  
  
  
  
• f: progressive, entity-selective clipping allows peeling the structural model of HIV layer by layer
• (explore interactively)

• The transparency of each sphere varies dynamically depending on its orientation relative to the camera (apparently each sphere has a normal vector and becomes more transparent as that vector is oriented away from the viewpoint)
• a, b: exosome lipid membrane with opacity 0.6 and 0.1, respectively
• (explore interactively)
• c: post-synapse model in black with transparency 0.015
• (explore interactively)

Two modules underlie these capabilities (explore interactively):

• snapshots: ability to save the current state, including overall appearance, styles, colors, clipping, camera position, etc., for later restoration or sequential playback like a slideshow (basically analogous to a Chimera "scene")
  • Mol* facilitates automatic view interpolation between snapshots
  • a snapshot includes a text description
  
  
• labeling: 3D text that can be added to any part of the scene
  • links to other snapshots can be defined using Markdown (a, such that mouseover in b highlights proteins in green)
  • customizable as to font, color, border appearance, tethering mechanism, tooltip text

Examples of models available in Mesoscale Explorer, where names in blue indicate a guided tour.

Loading time (right axis, cyan lines) and rendering speed (left axis, histogram in primary colors) for various example models in default views with rocking animation, at three different levels of quality.

• support additional file formats
  
  
• develop advanced lighting enhancements such as global illumination
  
  
• investigate supporting automated tours à la Molecumentary
  
  
• implement better story editing à la ScrollyVis
  
  
• support dynamic data (trajectories) à la Simularium Viewer