toolshed: Manage installed and available tools

The Toolshed provides an interface for finding installed bundles as well as bundles available for installation from a remote server. The Toolshed can handle updating, installing and uninstalling bundles while taking care of inter-bundle dependencies.

The Toolshed interface uses pkg_resources heavily.

Each Python distribution, a ChimeraX Bundle, may contain multiple tools, commands, data formats, and specifiers, with metadata entries for each deliverable.

In addition to the normal Python package metadta, The ‘ChimeraX’ classifier entries give additional information. Depending on the values of ‘ChimeraX’ metadata fields, modules need to override methods of the BundleAPI class. Each bundle needs a ‘ChimeraX :: Bundle’ entry that consists of the following fields separated by double colons (::).

  1. ChimeraX :: Bundlestr constant

    Field identifying entry as bundle metadata.

  2. categoriesstr

    Comma-separated list of categories in which the bundle belongs.

  3. session_versionstwo comma-separated integers

    Minimum and maximum session version that the bundle can read.

  4. supercedes : str Comma-separated list of superceded bundle names.

  5. custom_initstr

    Whether bundle has initialization code that must be called when ChimeraX starts. Either ‘true’ or ‘false’. If ‘true’, the bundle must override the BundleAPI’s ‘initialize’ and ‘finish’ functions.

Bundles that provide tools need:

  1. ChimeraX :: Toolstr constant

    Field identifying entry as tool metadata.

  2. tool_namestr

    The globally unique name of the tool (also shown on title bar).

  3. categoriesstr

    Comma-separated list of categories in which the tool belongs. Should be a subset of the bundle’s categories.

  4. synopsisstr

    A short description of the tool. It is here for uninstalled tools, so that users can get more than just a name for deciding whether they want the tool or not.

Tools are created via the bundle’s ‘start_tool’ function. Bundles may provide more than one tool.

Bundles that provide commands need:

  1. ChimeraX :: Commandstr constant

    Field identifying entry as command metadata.

  2. command namestr

    The (sub)command name. Subcommand names have spaces in them.

  3. categoriesstr

    Comma-separated list of categories in which the command belongs. Should be a subset of the bundle’s categories.

  4. synopsisstr

    A short description of the command. It is here for uninstalled commands, so that users can get more than just a name for deciding whether they want the command or not.

Commands are lazily registered, so the argument specification isn’t needed until the command is first used. Bundles may provide more than one command.

Bundles that provide selectors need:

  1. ChimeraX :: Selectorstr constant

    Field identifying entry as command metadata.

  2. selector namestr

    The selector’s name.

  3. synopsisstr

    A short description of the selector. It is here for uninstalled selectors, so that users can get more than just a name for deciding whether they want the selector or not.

4: atomicstr

An optional boolean specifying whether the selector applies to atoms and bonds. Defaults to ‘true’ and should be set to ‘false’ if selector should not appear in Basic Actions tool, e.g., showing/hiding selected items does nothing.

Commands are lazily registered, so the argument specification isn’t needed until the command is first used. Bundles may provide more than one command.

Bundles that provide data formats need:

  1. ChimeraX :: DataFormatstr constant

    Field identifying entry as data format metadata.

  2. data_namestr

    The name of the data format.

  3. nicknamesstr

    An optional comma-separated list of alternative names. Often a short name is provided. If not provided, it defaults to the lowercase version of the data format name.

  4. categorystr

    The toolshed category.

  5. suffixesstr

    An optional comma-separated list of strings with leading periods, e.g., ‘.pdb’.

  6. mime_typesstr

    An optinal comma-separated list of strings, e.g., ‘chemical/x-pdb’.

  7. urlstr

    A string that has a URL that points to the data format’s documentation.

  8. dangerousstr

    An optional boolean and should be ‘true’ if the data format is insecure – defaults to true if a script.

  9. iconstr

    An optional string containing the filename of the icon – it defaults to the default icon for the category. The file should be ?TODO? – metadata dir? package dir?

  10. synopsis : str A short description of the data format. It is here because it needs to be part of the metadata available for uninstalled data format, so that users can get more than just a name for deciding whether they want the data format or not.

Bundles may provide more than one data format. The data format metadata includes everything needed for the Mac OS X application property list.

Data formats that can be fetched:

# ChimeraX :: Fetch :: database_name :: format_name :: prefixes :: example_id :: is_default

Data formats that can be opened:

# ChimeraX :: Open :: format_name :: tag :: is_default

Data formats that can be saved:

# ChimeraX :: Save :: format_name :: tag :: is_default

Bundles that have other data:

# ChimeraX :: DataDir :: dir_path # ChimeraX :: IncludeDir :: dir_path # ChimeraX :: LibraryDir :: dir_path

Attributes

TOOLSHED_BUNDLE_INFO_ADDEDstr

Name of trigger fired when new bundle metadata is registered. The trigger data is a BundleInfo instance.

TOOLSHED_BUNDLE_INSTALLEDstr

Name of trigger fired when a new bundle is installed. The trigger data is a BundleInfo instance.

TOOLSHED_BUNDLE_UNINSTALLEDstr

Name of trigger fired when an installed bundle is removed. The trigger data is a BundleInfo instance.

TOOLSHED_BUNDLE_INFO_RELOADEDstr

Name of trigger fired when bundle metadata is reloaded. The trigger data is a BundleInfo instance.

Notes

The term ‘installed’ refers to bundles whose corresponding Python module or package is installed on the local machine. The term ‘available’ refers to bundles that are listed on a remote server but have not yet been installed on the local machine.

class BundleAPI

Bases: object

API for accessing bundles

The metadata for the bundle indicates which of the methods need to be implemented.

static data_dir(bundle_info)

Supported API . Returns path to directory of bundle-specific data.

Used to get directory path to data included in the bundle.

Parameters
bundle_infoBundleInfo instance.
Returns
str or None
static executable_dir(bundle_info)

Experimental API . Returns path to directory of compiled executables.

Used to get directory path to executables at run-time.

Parameters
bundle_infoBundleInfo instance.
Returns
str or None
static fetch_from_database(session, identifier, **kw)

Supported API . Called to fetch an entry from a network resource.

Arguments and return values are as described for save functions in chimerax.core.fetch. The format name will be in the format_name keyword. Whether a cache may be used will be in the ignore_cache keyword.

static finish(session, bundle_info)

Supported API . Called to deinitialize a bundle in a session.

Must be defined if the custom_init metadata field is set to ‘true’. finish is called when the bundle is unloaded.

Parameters
sessionSession instance.
bundle_infoBundleInfo instance.
static get_class(name)

Supported API . Called to get named class from bundle.

Used when restoring sessions. Instances whose class can’t be found via ‘get_class’ can not be saved in sessions. And those classes must implement the State API.

Parameters
namestr

Name of class in bundle.

static include_dir(bundle_info)

Experimental API . Returns path to directory of C++ header files.

Used to get directory path to C++ header files needed for compiling against libraries provided by the bundle.

Parameters
bundle_infoBundleInfo instance.
Returns
str or None
static init_manager(session, bundle_info, name, **kw)

Supported API . Called to create and return a manager in a bundle.

Must be defined if there is a Manager tag in the bundle. init_manager is called when bundles are first loaded. It is the responsibility of init_manager to make the manager locatable, e.g., assign as an attribute of session.

Parameters
sessionSession instance.
bundle_infoBundleInfo instance.
namestr.

Name of manager to initialize.

kwkeyword arguments.

Keyword arguments listed in the bundle_info.xml.

Returns
ProviderManager instance

The created manager.

static initialize(session, bundle_info)

Supported API . Called to initialize a bundle in a session.

Must be defined if the custom_init metadata field is set to ‘true’. initialize is called when the bundle is first loaded. To make ChimeraX start quickly, custom initialization is discouraged.

Parameters
sessionSession instance.
bundle_infoBundleInfo instance.
static library_dir(bundle_info)

Experimental API . Returns path to directory of compiled libraries.

Used to get directory path to libraries (shared objects, DLLs) for linking against libraries provided by the bundle.

Parameters
bundle_infoBundleInfo instance.
Returns
str or None
static open_file(session, stream_or_path, optional_format_name, optional_file_name, **kw)

Supported API . Called to open a file.

Second arg must be ‘stream’ or ‘path’. Depending on the name, either an open data stream or a filesystem path will be provided. The third and fourth arguments are optional (remove optional_ from their names if you provide them). ‘format-name’ will be the first nickname of the format if it has any, otherwise the full format name, but all lower case. ‘file_name’ if the name of input file, with path and compression suffix components stripped.

You shouldn’t actually use ‘kw’ but instead use the actual keyword args that your format declares that it accepts (in its bundle_info.xml file).

Returns
tuple

The return value is a 2-tuple whose first element is a list of Model instances and second element is a string containing a status message, such as the number of atoms and bonds found in the open models.

static register_command(*args)

Supported API . When ChimeraX starts, it registers placeholders for commands from all bundles. When a command from this bundle is actually used, ChimeraX calls this method to register the function that implements the command functionality, and then calls the command function. On subsequent uses of the command, ChimeraX will call the command function directly instead of calling this method.

Parameters
bundle_infoBundleInfo instance.
command_infoCommandInfo instance.
loggerLogger instance.

Version 1 of the API pass in information for both the command to be registered and the bundle where it was defined.

commandstr
loggerLogger instance.

Version 0 of the API only passes in the name of the command to be registered.

static register_selector(*args)

Supported API . This method is called the first time when the selector is used.

Parameters
bundle_infoBundleInfo instance.
selector_infoSelectorInfo instance.
loggerchimerax.core.logger.Logger instance.

Version 1 of the API passes in information about both the selector to be registered and the bundle where it is defined.

selector_namestr
loggerchimerax.core.logger.Logger instance.

Version 0 of the API only passes in the name of the selector to be registered.

static run_provider(session, name, mgr, **kw)

Supported API . Called to invoke a provider in a bundle.

Must be defined if there is a Provider tag in the bundle. run_provider is called by the associated manager to perform the corresponding task.

Parameters
sessionSession instance.
namestr.

Name of provider to initialize.

mgrstr.

Name of manager for this provider.

kwkeyword arguments.

Keyword arguments listed in the bundle_info.xml.

static save_file(session, stream, name, **kw)

Supported API . Called to save a file.

Arguments and return values are as described for save functions in chimerax.core.io. The format name will be in the format_name keyword.

static start_tool(*args)

Supported API . This method is called when the tool is invoked, typically from the application menu. Errors should be reported via exceptions.

Parameters
sessionchimerax.core.session.Session instance.
bundle_infoBundleInfo instance.
tool_infoToolInfo instance.

Version 1 of the API passes in information for both the tool to be started and the bundle where it was defined.

sessionchimerax.core.session.Session instance.
tool_namestr.

Version 0 of the API only passes in the name of the tool to be started.

Returns
ToolInstance instance

The created tool.

class ProviderManager

Bases: object

API for managers created by bundles

Managers returned by bundle init_manager methods should be an instance of this class.

abstract add_provider(self, bundle_info, provider_name, **kw)

Experimental API . Callback invoked to add provider to this manager.

Parameters
sessionchimerax.core.session.Session instance.
bundle_infoBundleInfo instance.
provider_namestr.
end_providers(self)

Experimental API . Callback invoked after all providers have been added.

class Toolshed(logger, rebuild_cache=False, check_remote=False, remote_url=None, check_available=True)

Bases: object

Toolshed keeps track of the list of bundle metadata, aka BundleInfo.

Tool metadata may be for “installed” bundles, where their code is already downloaded from the remote server and installed locally, or “available” bundles, where their code is not locally installed.

Attributes
triggersTriggerSet instance

Where to register handlers for toolshed triggers

bootstrap_bundles(self, session, safe_mode)

Supported API . Do custom initialization for installed bundles

After adding the Toolshed singleton to a session, allow bundles need to install themselves into the session, (For symmetry, there should be a way to uninstall all bundles before a session is discarded, but we don’t do that yet.)

bundle_info(self, logger, installed=True, available=False)

Supported API . Return list of bundle info.

Parameters
installedboolean

True to include installed bundle metadata in return value; False otherwise

availableboolean

True to include available bundle metadata in return value; False otherwise

Returns
list of BundleInfo instances

Combined list of all selected types of bundle metadata.

find_bundle(self, name, logger, installed=True, version=None)

Supported API . Return a BundleInfo instance with the given name.

Parameters
namestr

Name (internal or display name) of the bundle of interest.

loggerLogger instance

Logging object where warning and error messages are sent.

installedboolean

True to check only for installed bundles; False otherwise.

versionstr

None to find any version; specific string to check for one particular version.

find_bundle_for_class(self, cls)

Supported API . Find bundle that has given class

find_bundle_for_command(self, cmd)

Supported API . Find bundle registering given command

cmd must be the full command name, not an abbreviation.

find_bundle_for_tool(self, name)

Supported API . Find named tool and its bundle

Return the bundle it is in and its true name.

import_bundle(self, bundle_name, logger, install='ask', session=None)

Supported API . Return the module for the bundle with the given name.

Parameters
bundle_namestr

Name (internal or display name) of the bundle of interest.

loggerLogger instance

Logging object where warning and error messages are sent.

install: str

Action to take if bundle is uninstalled but available. “ask” (default) means to ask user, if session is not None; “never” means not to install; and “always” means always install.

sessionchimerax.core.session.Session instance.

Session that is requesting the module. Defaults to None.

Raises
ImportError

Raised if a module for the bundle cannot be found.

import_package(self, package_name, logger, install=None, session=None)

Experimental API . Return package of given name if it is associated with a bundle.

Parameters
module_namestr

Name of the module of interest.

loggerLogger instance

Logging object where warning and error messages are sent.

install: str

Action to take if bundle is uninstalled but available. “ask” (default) means to ask user, if session is not None; “never” means not to install; and “always” means always install.

sessionchimerax.core.session.Session instance.

Session that is requesting the module. Defaults to None.

Raises
ImportError

Raised if a module for the bundle cannot be found.

install_bundle(self, bundle, logger, *, per_user=True, reinstall=False, session=None)

Supported API . Install the bundle by retrieving it from the remote shed.

Parameters
bundlestring or BundleInfo instance

If string, path to wheel installer. If instance, should be from the available bundle list.

per_userboolean

True to install bundle only for the current user (default); False to install for everyone.

reinstallboolean

True to force reinstall package.

loggerLogger instance

Logging object where warning and error messages are sent.

Raises
ToolshedInstalledError

Raised if the bundle is already installed.

Notes

A TOOLSHED_BUNDLE_INSTALLED trigger is fired after installation.

reload(self, logger, *, session=None, reread_cache=True, rebuild_cache=False, check_remote=False, report=False)

Supported API . Discard and reread bundle info.

Parameters
loggerLogger instance

A logging object where warning and error messages are sent.

rebuild_cacheboolean

True to ignore local cache of installed bundle information and rebuild it by scanning Python directories; False otherwise.

check_remoteboolean

True to check remote server for updated information; False to ignore remote server

uninstall_bundle(self, bundle, logger, *, session=None)

Supported API . Uninstall bundle by removing the corresponding Python distribution.

Parameters
bundlestring or BundleInfo instance

If string, path to wheel installer. If instance, should be from the available bundle list.

loggerLogger instance

Logging object where warning and error messages are sent.

Raises
ToolshedInstalledError

Raised if the bundle is not installed.

Notes

A TOOLSHED_BUNDLE_UNINSTALLED trigger is fired after package removal.

exception ToolshedError

Bases: Exception

Generic Toolshed error.

exception ToolshedInitializationError

Bases: ToolshedError

Initialization error.

This exception derives from ToolshedError and is usually raised when doing manager, provider or custom initialization.

exception ToolshedInstalledError

Bases: ToolshedError

Bundle-already-installed error.

This exception derives from ToolshedError and is usually raised when trying to install a bundle that is already installed or to uninstall a bundle that is not installed yet.

exception ToolshedUnavailableError

Bases: ToolshedError

Bundle-not-found error.

This exception derives from ToolshedError and is usually raised when no Python distribution can be found for a bundle.

get_toolshed()

Supported API . Return current toolshed.

Returns
Toolshed instance

The toolshed singleton.

The toolshed singleton will be None if py:func:init hasn’t been called yet.
init(*args, debug=None, **kw)

Supported API . Initialize toolshed.

The toolshed instance is a singleton across all sessions. The first call creates the instance and all subsequent calls return the same instance. The toolshed debugging state is updated at each call.

Parameters
debugboolean

If true, debugging messages are sent to standard output. Default value is false.

other argumentsany

All other arguments are passed to the Toolshed initializer.

class ToolInfo(name, categories, synopsis=None)

Bases: object

Metadata about a tool

Attributes
namestr

Tool name.

categorieslist of str

Menu categories that tool should be in.

synopsisstr

One line description.

class BundleInfo(name, installed, version=None, api_package_name=None, categories=(), synopsis=None, description='Unknown', session_versions=range(1, 2), custom_init=False, data_dir=None, include_dir=None, library_dir=None, executable_dir=None, managers=None, providers=None, inits=None, packages=[], supercedes=[])

Bases: object

Supported API. Metadata about a bundle, whether installed or available.

A BundleInfo instance stores the properties about a bundle and can create a tool instance.

Attributes
commandslist of CommandInfo

List of commands registered for this bundle.

toolslist of ToolInfo

List of tools registered for this bundle.

installedboolean

True if this bundle is installed locally; False otherwise.

file_formatslist of DataInfo

List of data formats that this bundle knows about.

session_versionsrange

Given as the minimum and maximum session versions that this bundle can read.

session_write_versioninteger

The session version that bundle data is written in. Defaults to maximum of ‘session_versions’.

custom_initboolean

Whether bundle has custom initialization code

namereadonly str

Supported API.

synopsisreadonly str

Supported API.

versionreadonly str

Supported API.

data_dir:

Path (relative to bundle root) of directory of data files

include_dir:

Path (relative to bundle root) of directory of compilation include files

library_dir:

Path (relative to bundle root) of directory of link libraries

executable_dir:

Path (relative to bundle root) of directory of executables

cache_data(self)

Supported API . Return state data that can be used to recreate the instance.

Returns
2-tuple of (list, dict)

List and dictionary suitable for passing to BundleInfo.

data_dir(self)

Supported API . Return bundle data directory.

dependents(self, logger)

Supported API . Return set of bundles that directly depends on this one.

Parameters
loggerLogger instance

Where to log error messages.

Returns
set of BundleInfo instances

Dependent bundles.

deregister(self, logger)

Supported API . Deregister bundle commands, tools, data formats, selectors, etc.

Parameters
loggerLogger instance

Where to log error messages.

distribution(self)

Supported API . Return distribution information.

Returns
2-tuple of (str, str).

Distribution name and version.

executable_dir(self)

Supported API . Return bundle executable directory.

finish(self, session)

Supported API . Deinitialize bundle by calling custom finish code if needed.

This method is only called when a bundle is explicitly unloaded. In particular, it is not called when ChimeraX exits normally.

classmethod from_cache_data(data)

Supported API . Class method for reconstructing instance from cache data. Returns ——- instance of BundleInfo

get_class(self, class_name, logger)

Supported API . Return bundle’s class with given name.

get_module(self, force_import=True)

Supported API . Return module that has bundle’s code

include_dir(self)

Supported API . Return bundle include directory.

init_manager(self, session, name, **kw)

Supported API . Initialize bundle manager if needed.

initialize(self, session)

Supported API . Initialize bundle by calling custom initialization code if needed.

library_dir(self)

Supported API . Return bundle library directory.

property name

Supported API. Return bundle name.

newer_than(self, bi)

Supported API . Return whether this BundleInfo instance is newer than given one

Parameters
biBundleInfo instance

The instance to compare against

Returns
Boolean

True if this instance is newer; False if ‘bi’ is newer.

register(self, logger)

Supported API . Register bundle commands, tools, data formats, selectors, etc.

Parameters
loggerLogger instance

Where to log error messages.

run_provider(self, session, name, mgr, **kw)

Supported API . Called by manager to invoke bundle provider.

start_tool(self, session, tool_name, *args, **kw)

Supported API . Create and return a tool instance.

Parameters
sessionSession instance

The session in which the tool will run.

argsany

Positional arguments to pass to tool instance initializer.

kwany

Keyword arguments to pass to tool instance initializer.

Returns
ToolInstance instance

The registered running tool instance.

Raises
ToolshedUninstalledError

If the bundle is not installed.

ToolshedError

If the tool cannot be started.

property synopsis

Supported API. Return bundle synopsis.

unload(self, logger)

Supported API . Unload bundle modules (as best as we can).

property version

Supported API. Return bundle version.

class CommandInfo(name, categories, synopsis=None)

Bases: ToolInfo

Metadata about a command

class SelectorInfo(name, synopsis=None, atomic=True)

Bases: ToolInfo

Metadata about a selector

Attributes
namestr

Tool name.

synopsisstr

One line description.

atomicboolean

Whether selector applies to atoms and bonds.