toolshed: Manage installed and available tools

The Toolshed provides an interface for finding installed bundles as well as bundles available for installation from a remote server. The Toolshed can handle updating, installing and uninstalling bundles while taking care of inter-bundle dependencies.

The Toolshed interface uses distlib heavily. For example, Distribution instances from distlib are tracked for both available and installed bundles; the distlib.locators.Locator class is used for finding an installed distlib.database.Distribution.

Each Python distribution, a ChimeraX Bundle, (ChimeraX uses distlib.wheel.Wheel) may contain multiple tools, commands, data formats, and specifiers, with metadata entries for each deliverable.

In addition to the normal Python package metadta, The ‘ChimeraX’ classifier entries give additional information. Depending on the values of ‘ChimeraX’ metadata fields, modules need to override methods of the BundleAPI class. Each bundle needs a ‘ChimeraX :: Bundle’ entry that consists of the following fields separated by double colons (::).

  1. ChimeraX :: Bundle : str constant
    Field identifying entry as bundle metadata.
  2. categories : str
    Comma-separated list of categories in which the bundle belongs.
  3. session_versions : two comma-separated integers
    Minimum and maximum session version that the bundle can read.
  4. supercedes : str Comma-separated list of superceded bundle names.
  5. custom_init : str
    Whether bundle has initialization code that must be called when ChimeraX starts. Either ‘true’ or ‘false’. If ‘true’, the bundle must override the BundleAPI’s ‘initialize’ and ‘finish’ functions.

Bundles that provide tools need:

  1. ChimeraX :: Tool : str constant
    Field identifying entry as tool metadata.
  2. tool_name : str
    The globally unique name of the tool (also shown on title bar).
  3. categories : str
    Comma-separated list of categories in which the tool belongs. Should be a subset of the bundle’s categories.
  4. synopsis : str
    A short description of the tool. It is here for uninstalled tools, so that users can get more than just a name for deciding whether they want the tool or not.

Tools are created via the bundle’s ‘start_tool’ function. Bundles may provide more than one tool.

Bundles that provide commands need:

  1. ChimeraX :: Command : str constant
    Field identifying entry as command metadata.
  2. command name : str
    The (sub)command name. Subcommand names have spaces in them.
  3. categories : str
    Comma-separated list of categories in which the command belongs. Should be a subset of the bundle’s categories.
  4. synopsis : str
    A short description of the command. It is here for uninstalled commands, so that users can get more than just a name for deciding whether they want the command or not.

Commands are lazily registered, so the argument specification isn’t needed until the command is first used. Bundles may provide more than one command.

Bundles that provide selectors need:

  1. ChimeraX :: Selector : str constant
    Field identifying entry as command metadata.
  2. selector name : str
    The selector’s name.
  3. synopsis : str
    A short description of the selector. It is here for uninstalled selectors, so that users can get more than just a name for deciding whether they want the selector or not.

Commands are lazily registered, so the argument specification isn’t needed until the command is first used. Bundles may provide more than one command.

Bundles that provide data formats need:

  1. ChimeraX :: DataFormat : str constant
    Field identifying entry as data format metadata.
  2. data_name : str
    The name of the data format.
  3. nicknames : str
    An optional comma-separated list of alternative names. Often a short name is provided. If not provided, it defaults to the lowercase version of the data format name.
  4. category : str
    The toolshed category.
  5. suffixes : str
    An optional comma-separated list of strings with leading periods, e.g., ‘.pdb’.
  6. mime_types : str
    An optinal comma-separated list of strings, e.g., ‘chemical/x-pdb’.
  7. url : str
    A string that has a URL that points to the data format’s documentation.
  8. dangerous : str
    An optional boolean and should be ‘true’ if the data format is insecure – defaults to true if a script.
  9. icon : str
    An optional string containing the filename of the icon – it defaults to the default icon for the category. The file should be ?TODO? – metadata dir? package dir?
  10. synopsis : str A short description of the data format. It is here because it needs to be part of the metadata available for uninstalled data format, so that users can get more than just a name for deciding whether they want the data format or not.

Bundles may provide more than one data format. The data format metadata includes everything needed for the Mac OS X application property list.

Data formats that can be fetched:

# ChimeraX :: Fetch :: database_name :: format_name :: prefixes :: example_id :: is_default

Data formats that can be opened:

# ChimeraX :: Open :: format_name :: tag :: is_default

Data formats that can be saved:

# ChimeraX :: Save :: format_name :: tag :: is_default

Attributes

TOOLSHED_BUNDLE_INFO_ADDED : str
Name of trigger fired when new bundle metadata is registered. The trigger data is a BundleInfo instance.
TOOLSHED_BUNDLE_INSTALLED : str
Name of trigger fired when a new bundle is installed. The trigger data is a BundleInfo instance.
TOOLSHED_BUNDLE_UNINSTALLED : str
Name of trigger fired when an installed bundle is removed. The trigger data is a BundleInfo instance.
TOOLSHED_BUNDLE_INFO_RELOADED : str
Name of trigger fired when bundle metadata is reloaded. The trigger data is a BundleInfo instance.

Notes

The term ‘installed’ refers to bundles whose corresponding Python module or package is installed on the local machine. The term ‘available’ refers to bundles that are listed on a remote server but have not yet been installed on the local machine.

class BundleAPI

Bases: object

API for accessing bundles

The metadata for the bundle indicates which of the methods need to be implemented.

static fetch_from_database(identifier, **kw)

Experimental API . Called to fetch an entry from a network resource.

Arguments and return values are as described for save functions in chimerax.core.fetch. The format name will be in the format_name keyword. Whether a cache may be used will be in the ignore_cache keyword.

static finish(bundle_info)

Experimental API . Called to deinitialize a bundle in a session.

Parameters:

session : Session instance.

bundle_info : BundleInfo instance.

Must be defined if the ``custom_init`` metadata field is set to ‘true’.

``finish`` is called when the bundle is unloaded.

static get_class()

Experimental API . Called to get named class from bundle.

Parameters:

name : str

Name of class in bundle.

Used when restoring sessions. Instances whose class can’t be found via

‘get_class’ can not be saved in sessions. And those classes must implement

the :py:class:`~chimerax.core.state.State` API.

static initialize(bundle_info)

Experimental API . Called to initialize a bundle in a session.

Parameters:

session : Session instance.

bundle_info : BundleInfo instance.

Must be defined if the ``custom_init`` metadata field is set to ‘true’.

``initialize`` is called when the bundle is first loaded.

To make ChimeraX start quickly, custom initialization is discouraged.

static open_file(stream_or_path, optional_format_name, optional_file_name, **kw)

Experimental API . Called to open a file.

Second arg must be ‘stream’ or ‘path’. Depending on the name, either an open data stream or a filesystem path will be provided. The third and fourth arguments are optional (remove optional_ from their names if you provide them). ‘format-name’ will be the first nickname of the format if it has any, otherwise the full format name, but all lower case. ‘file_name’ if the name of input file, with path and compression suffix components stripped.

You shouldn’t actually use ‘kw’ but instead use the actual keyword args that your format declares that it accepts (in its bundle_info.xml file).

Returns:

tuple

The return value is a 2-tuple whose first element is a list of Model instances and second element is a string containing a status message, such as the number of atoms and bonds found in the open models.

static register_command()

Experimental API . When ChimeraX starts, it registers placeholders for commands from all bundles. When a command from this bundle is actually used, ChimeraX calls this method to register the function that implements the command functionality, and then calls the command function. On subsequent uses of the command, ChimeraX will call the command function directly instead of calling this method.

Parameters:

bundle_info : instance of BundleInfo

command_info : instance of CommandInfo

logger : Logger instance.

Version 1 of the API pass in information for both the command to be registered and the bundle where it was defined.

command : str

logger : Logger instance.

Version 0 of the API only passes in the name of the command to be registered.

static register_selector()

Experimental API . This method is called the first time when the selector is used.

Parameters:

bundle_info : instance of BundleInfo

selector_info : instance of SelectorInfo

logger : chimerax.core.logger.Logger instance.

Version 1 of the API passes in information about both the selector to be registered and the bundle where it is defined.

selector_name : str

logger : chimerax.core.logger.Logger instance.

Version 0 of the API only passes in the name of the selector to be registered.

static save_file(stream, name, **kw)

Experimental API . Called to save a file.

Arguments and return values are as described for save functions in chimerax.core.io. The format name will be in the format_name keyword.

static start_tool()

Experimental API . This method is called when the tool is invoked, typically from the application menu. Errors should be reported via exceptions.

Parameters:

session : chimerax.core.session.Session instance.

bundle_info : instance of BundleInfo

tool_info : instance of ToolInfo

Version 1 of the API passes in information for both the tool to be started and the bundle where it was defined.

session : chimerax.core.session.Session instance.

tool_name : str.

Version 0 of the API only passes in the name of the tool to be started.

Returns:

ToolInstance instance

The created tool.

class Toolshed(logger, rebuild_cache=False, check_remote=False, remote_url=None, check_available=True)

Bases: object

Toolshed keeps track of the list of bundle metadata, aka BundleInfo.

Tool metadata may be for “installed” bundles, where their code is already downloaded from the remote server and installed locally, or “available” bundles, where their code is not locally installed.

Attributes

triggers (TriggerSet instance) Where to register handlers for toolshed triggers
bootstrap_bundles(session)

Experimental API . Do custom initialization for installed bundles

After adding the Toolshed singleton to a session, allow bundles need to install themselves into the session, (For symmetry, there should be a way to uninstall all bundles before a session is discarded, but we don’t do that yet.)

bundle_info(logger, installed=True, available=False)

Experimental API . Return list of bundle info.

Parameters:

installed : boolean

True to include installed bundle metadata in return value; False otherwise

available : boolean

True to include available bundle metadata in return value; False otherwise

Returns:

list of BundleInfo instances

Combined list of all selected types of bundle metadata.

find_bundle(name, logger, installed=True, version=None)

Experimental API . Return a BundleInfo instance with the given name.

Parameters:

name : str

Name (internal or display name) of the bundle of interest.

installed : boolean

True to check only for installed bundles; False otherwise.

version : str

None to find any version; specific string to check for one particular version.

find_bundle_for_class(cls)

Experimental API . Find bundle that has given class

find_bundle_for_command(cmd)

Experimental API . Find bundle registering given command

cmd must be the full command name, not an abbreviation.

find_bundle_for_tool(name)

Experimental API . Find named tool and its bundle

Return the bundle it is in and its true name.

install_bundle(bundle, logger, *, per_user=True, reinstall=False, session=None)

Experimental API . Install the bundle by retrieving it from the remote shed.

Parameters:

bundle : string or BundleInfo instance

If string, path to wheel installer. If instance, should be from the available bundle list.

per_user : boolean

True to install bundle only for the current user (default); False to install for everyone.

reinstall : boolean

True to force reinstall package.

logger : Logger instance

Logging object where warning and error messages are sent.

Raises:

ToolshedInstalledError

Raised if the bundle is already installed.

Notes

A TOOLSHED_BUNDLE_INSTALLED trigger is fired after installation.

reload(logger, *, session=None, reread_cache=True, rebuild_cache=False, check_remote=False, report=False)

Experimental API . Discard and reread bundle info.

Parameters:

logger : Logger instance

A logging object where warning and error messages are sent.

rebuild_cache : boolean

True to ignore local cache of installed bundle information and rebuild it by scanning Python directories; False otherwise.

check_remote : boolean

True to check remote server for updated information; False to ignore remote server; None to use setting from user preferences.

set_install_timestamp(per_user=False)

Experimental API . Set last installed timestamp.

uninstall_bundle(bundle, logger, *, session=None)

Experimental API . Uninstall bundle by removing the corresponding Python distribution.

Parameters:

bundle : string or BundleInfo instance

If string, path to wheel installer. If instance, should be from the available bundle list.

logger : Logger instance

Logging object where warning and error messages are sent.

Raises:

ToolshedInstalledError

Raised if the bundle is not installed.

Notes

A TOOLSHED_BUNDLE_UNINSTALLED trigger is fired after package removal.

exception ToolshedError

Bases: Exception

Generic Toolshed error.

exception ToolshedInstalledError

Bases: ToolshedError

Bundle-already-installed error.

This exception derives from ToolshedError and is usually raised when trying to install a bundle that is already installed or to uninstall a bundle that is not installed yet.

exception ToolshedUnavailableError

Bases: ToolshedError

Bundle-not-found error.

This exception derives from ToolshedError and is usually raised when no Python distribution can be found for a bundle.

init(*args, debug=None, **kw)

Experimental API . Initialize toolshed.

The toolshed instance is a singleton across all sessions. The first call creates the instance and all subsequent calls return the same instance. The toolshed debugging state is updated at each call.

Parameters:

debug : boolean

If true, debugging messages are sent to standard output. Default value is false.

other arguments : any

All other arguments are passed to the Toolshed initializer.

Returns:

Toolshed instance

The toolshed singleton.

class ToolInfo(name, categories, synopsis=None)

Bases: object

Metadata about a tool

Attributes

name (str) Tool name.
categories (list of str) Menu categories that tool should be in.
synopsis (str) One line description.
class BundleInfo(name, installed, version=None, api_package_name=None, categories=(), synopsis=None, description='Unknown', session_versions=range(1, 2), custom_init=False, packages=[], supercedes=[])

Bases: object

Metadata about a bundle, whether installed or available.

A BundleInfo instance stores the properties about a bundle and can create a tool instance.

Attributes

commands (list of CommandInfo) List of commands registered for this bundle.
installed (boolean) True if this bundle is installed locally; False otherwise.
file_formats (list of DataInfo) List of data formats that this bundle knows about.
session_versions (range) Given as the minimum and maximum session versions that this bundle can read.
session_write_version (integer) The session version that bundle data is written in. Defaults to maximum of ‘session_versions’.
custom_init (boolean) Whether bundle has custom initialization code
name (readonly str) The internal name of the bundle.
synopsis (readonly str) Short description of this bundle.
version (readonly str) Bundle version (which is actually the same as the distribution version, so all bundles from the same distribution share the same version).
cache_data()

Experimental API . Return state data that can be used to recreate the instance.

Returns:

2-tuple of (list, dict)

List and dictionary suitable for passing to BundleInfo.

dependents(logger)

Experimental API . Return set of bundles that directly depends on this one.

Parameters:

logger : Logger instance

Where to log error messages.

Returns:

set of BundleInfo instances

Dependent bundles.

deregister(logger)

Experimental API . Deregister bundle commands, tools, data formats, selectors, etc.

Parameters:

logger : Logger instance

Where to log error messages.

distribution()

Experimental API . Return distribution information.

Returns:

2-tuple of (str, str).

Distribution name and version.

finish(session)

Experimental API . Deinitialize bundle by calling custom finish code if needed.

get_class(class_name, logger)

Experimental API . Return bundle’s class with given name.

get_module()

Experimental API . Return module that has bundle’s code

initialize(session)

Experimental API . Initialize bundle by calling custom initialization code if needed.

newer_than(bi)

Experimental API . Return whether this BundleInfo instance is newer than given one

Parameters:

bi : BundleInfo instance

The instance to compare against

Returns:

Boolean

True if this instance is newer; False if ‘bi’ is newer.

register(logger)

Experimental API . Register bundle commands, tools, data formats, selectors, etc.

Parameters:

logger : Logger instance

Where to log error messages.

register_available_commands(logger)

Experimental API . Register available commands with cli.

start_tool(session, tool_name, *args, **kw)

Experimental API . Create and return a tool instance.

Parameters:

session : Session instance

The session in which the tool will run.

args : any

Positional arguments to pass to tool instance initializer.

kw : any

Keyword arguments to pass to tool instance initializer.

Returns:

ToolInstance instance

The registered running tool instance.

Raises:

ToolshedUninstalledError

If the bundle is not installed.

ToolshedError

If the tool cannot be started.

unload(logger)

Experimental API . Unload bundle modules (as best as we can).

class CommandInfo(name, categories, synopsis=None)

Bases: ToolInfo

Metadata about a command

class SelectorInfo(name, synopsis=None)

Bases: ToolInfo

Metadata about a selector

Attributes

name (str) Tool name.
synopsis (str) One line description.