Galaxy toolsheds

Galaxy toolsheds

Relatively painless tool-sharing

This is a more recent innovation in Galaxy, which can make it a somewhat confused one: the concept of the toolshed has changed over its lifetime, the documentation is incomplete, and there's a slightly strange emphasis in the documentation that exists. So …

Mile-high description

Toolsheds are webapps just like Galaxy. They provides a way of sharing Galaxy tools, installing them into a Galaxy instance from within that instance (i.e. through the web), and being notified and obtaining updates to those tools. There's an official toolshed, a test one and you can set up your own.

Installing tools from the standard sheds

You need to be an administrator. Go into the admin menu and on the side you will see “Search and browse tool sheds”. Click this and you will get a list of the sheds available to you. Click on any of these and you get the choices for the individual repository: browse, search for tools, search for workflows. ** ** NOTE: an irritating feature of the way the sheds are set up is that as you descend into the shed, there's few explicit up/back links to take you to where you just were or a category higher. You'll have to use your back button. By search or browse, you'll end up on an index page listing tools. These will (strangely) give a button going to the tool, a description, the versions available and the owner/creator. Click on the button for one and you'll get a popup “Preview and install”. This is a complicated way of saying “go to the individual tool page”. Do that. The tool page will a bunch of information of the tool - actually tools if the package installs more than one. But the important button is in the top-right: “Install to Local Galaxy”. (the other button “Too shed asctions”, is a duplicate of the top level button giving you browse or search choices. It's an odd UI.) If you click the install button, you'll be offered the choice of putting the new tool(s) in a pre-existing section or a new one you create. Make a choice and then click “install” at the bottom of the page. After some minutes, it will appear in your instance. NOTE: downloaded tools are actually placed in a directory ../shed_tools, i.e. outside your actual galaxy directory. (Strange, but an illustration of the point that tools can actually live anywhere.) They're also stored as mercurial repositories.

Running your own toolshed

You might want to do this if you want to manage custom tools for your own Galaxy instance that you don't want to pass to the outside world. And it's dead easy. The toolshed app is included with the main Galaxy distribution and lives alongside it. Three key points:

  1. The configuration details for the toolshed live in community_wsgi.ini
  2. You run the toolshed with sh run_community.sh
  3. The toolshed uses a different database to the main Galaxy instance

Various of the necessary web configuration details can be found in the links below and are similar to those used by Galaxy. Some points worth highlighting:

  • port & host: should be set to where you want the toolshed to be available from
  • database_file: this is if you want to use the sqlite3 db. I had some problems with this, which were eventually solved by setting the path correctly for insertion in an sqlite protocol string (e.g./database/community.sqlite for a relative path setting the file in the database folder
  • file_path: again, had some problems with this, which were solved by creating the directory for the file (e.g. <galaxy inst>/database/community_files) explicitly
  • admin_users: like the main Galaxy installation, for setting the admins

One more thing must be done before users can upload tools to your shed: an admin has to create categories before any tools can be uploaded. Look under Admin / Browse catgeories. (Oddly, users don't actually have to assign a tool to categories when they create them, But there must be a category.)

Using a local toolshed

Here we mean “using” in terms of putting tools up for other Galaxy installations to download. You'll have to sign up to your toolshed to do this - it may live in the same directory as the main app, but it doesn't share data. Look on the toolbar under User / Register Now you have to create a create a repository to store your tool. Check out on the sidebar Create new repository. You'll have to enter a bunch of descriptive text / metadata. Afterwards, you'll have the choice of uploading files to your new repo. While you can upload bz2 and gzipped archives, which are unpacked automagically. (There's some confusion here, where the examples also indicates that tar is unpacked, which some opaque nesting rules.) You may run into an infuriating TypeError: array item must be char which is a problem that apparently occurs randomly, but is solved in more recent versions of the toolshed. Update your code. Your account that the toolshed is running under will also need credentials for using mercurial. In it's home directory, there'll be a .hgrc file, which should look something like this: [ui] username = My Name <my-name@example.com> [web] allow_archive = bz2, gz, zip The last line defines the archive types that you can download source code as. Confusingly, the toolshed will offer you the choice of downloading types that aren't allowed ("Download as [tar.gz|tar.bz2 ...]") and then spit back an error "Archive type is not allowed".

Further reading

Toolsheds have their wiki page, although it is strangely preoccupied with running your own toolshed and the technical underpinnings

Installing a tool manually

The easiest way to get a tool is by using a recent version of Galaxy and downloading from the toolshed. But sometimes it happens that a tool is not available in the toolshed, or that it is a toolshed you're not linked to, or that a more recent version is available from elsewhere. So you're going to have to manually install the tool from source. The steps are essentially:

  1. Obtain tarball/archive of tool
  2. Unpack
  3. Put folder in the tools folder of the galaxy instance
  4. Add the tool entry to the tool_conf.xml file

Manually downloading from toolshed

You can download a tool tarball from a toolshed, it's just a bit hidden.

  1. Go to the toolshed site of your choice
  2. Find the index listing containing the tool you are interested in (the page with a table headed “Name … Synopsis … Author”)
  3. Click on the tool name/button to get to the individual tool page
  4. Click the button in the upper right labelled “repository actions” and on the popup select “Download as a ” in the desired format

Configure tool

Add an entry in an appropriate place to the tool-conf.xml in the root of your galaxy instance. It should look something like this: <section name="Get Data" id="getext"> <tool file="data_source/upload.xml"/> <tool file="MY_TOOL_DIR/MY_TOOL_CONF.xml" /> <tool file="data_source/ucsc_tablebrowser.xml" /> ... where MY_TOOL_DIR is the name of the folder you placed in the tools dir and MY_TOOL_CONF.xml is the xml configuration file in that folder. See notes on tool development for further details. Restart your Galaxy instance to get it to pick up the new tool. If the tool does not show up in the right place on the tool menu, you've fucked up.

What it looks like in the end

So at the end your file hierarchy should look something like this: galaxy_dist (instance folder) - tool_conf.xml - tools - MY_TOOL_DIR - MY_TOOL_CONF.xml - (other tool files including executables)