Pipeline Guide

Foundry supports both Via and nf-core/Nextflow pipelines. This guide explains how to import each type into Foundry.

Importing Via Pipelines

To import Via pipelines, you must have admin permissions. Contact biocore@umassmed.edu for more details.

Go to the Pipelines page and click Create Pipeline.
Open the Import tab and click Import Via Pipeline.
Drag and drop one or more importable Foundry pipeline files (with .vf or .dn extensions).
Review the changes and confirm the import.
After the import completes, you will be redirected to the newly imported pipeline.

Resolving Conflicts

The Conflicts step lists the items in the file that clash with what this installation already holds. A process can clash in two ways.

The same process version, with different content. The file carries a version of a process that already exists here, but the content differs. The differences are shown under Conflict Detail, and there is nothing to choose: clicking Approve overwrites the version here with the one from the file.

A version id that is already taken. Every process version carries an id that means the same version on every installation, so a file cannot add a second copy of it. When that id already belongs to a version here that was removed, or to a version with different content under another process, the conflict is marked needs a decision and you pick what happens under What should happen. Nothing is preselected, and Go to Next Step stays disabled until every such conflict has a choice.

Choice	What it does	When it is offered
Keep the version already here	The imported pipeline uses the version this installation already holds. Nothing is written or deleted, and the version in the file is discarded. If the version here was removed, it stays removed.	Unless the version here was removed and its content differs from the file.
Bring back the removed version already here	Restores the removed version, which already holds the same content as the file. Nothing is overwritten.	Only when the version here was removed and holds exactly the same content as the file. Admins only.
Delete the version here and use the incoming one	Permanently deletes the version stored here and puts the version from the file in its place. This cannot be undone.	Only when nothing depends on the version here and it was not imported from a repository (see Processes Imported from a Repository). Admins only.

When Delete the version here and use the incoming one is not offered, it is still listed, greyed out, with the reason underneath. Anything that depends on the version here, such as pipelines that include it or runs that already used it, is listed next to Depended on by:. If none of the choices can be picked, the import cannot go ahead. Any user can pick Bring back the removed version already here or Delete the version here and use the incoming one in the dialog, but only an admin can complete an import that uses them; for anyone else, Approve is refused.

Processes Imported from a Repository

Not available in Foundry Connect 2.22.0 or earlier

This protection arrives in the release that follows 2.22.0.

A .vf import never overwrites or replaces a process version that was imported from a Git repository (see Importing Processes from a Repository). Foundry Connect stores such a process but does not execute it, and a .vf file does not record where a process came from, so overwriting or replacing the version here would turn it into an ordinary process that can run.

For a version id conflict, Delete the version here and use the incoming one is not offered, so the version here stays as it is.
For the same process version with different content, the import is refused when you click Approve, with a message naming the process and the repository it came from. Nothing from the file is imported.

Importing nf-core / Nextflow Pipelines

Foundry can import and run nf-core pipelines as well as other standard Nextflow pipelines.

Go to the Pipelines page and click Create Pipeline.
Open the Import tab.
Choose the Repository type: Public or Private.

a) Public Repository

Use this option if your pipeline repository is publicly accessible.

Select Public as the repository type.
Enter the Repository URL (for example, https://github.com/nf-core/sarek).
(Optional) Enter the tag/branch you want to use (for example, 3.6.1).
Set the Pipeline Type:
- Choose nf-core if this is an nf-core pipeline.
- Choose nextflow for other Nextflow pipelines.
Click Pull Pipeline to import it into Foundry.

b) Private Repository

Use this option if your pipeline repository is private.

Select Private as the repository type.
Choose the repository credentials you configured under Profile → Integration Tokens.
Enter the Repository URL (for example, https://github.com/nf-core/sarek).
(Optional) Enter the tag/branch you want to use (for example, 3.6.1).
Set the Pipeline Type:
- Choose nf-core if this is an nf-core pipeline.
- Choose nextflow for other Nextflow pipelines.
Click Pull Pipeline to import it into Foundry.

Standards for nf-core–like Pipelines

To ensure a smooth experience with nf-core or nf-core–like pipelines, we recommend following these standards.

1. Pipeline Input Schema (`nextflow_schema.json`)

For nf-core–style pipelines, Foundry relies on nextflow_schema.json to display pipeline inputs on the Run page.

You can refer to nf-core examples, such as:
Example inputs – nf-core/rnaseq nextflow_schema.json
If your pipeline doesn’t yet have a nextflow_schema.json file, you can create one using the
Pipeline Schema Builder Tool.

Important: Ensure that all inputs are organized within a group section.
In the schema builder, click Add Group and define parameters inside that group.

Example nextflow_schema.json structure:

{
  "$schema": "http://json-schema.org/draft-07/schema",
  "$id": "https://raw.githubusercontent.com/YOUR_PIPELINE/master/nextflow_schema.json",
  "title": "Nextflow pipeline parameters",
  "description": "This pipeline uses Nextflow and processes some kind of data. The JSON Schema was built using the nf-core pipeline schema builder.",
  "type": "object",
  "definitions": {
    "new_group_1": {
      "title": "New Group 1",
      "type": "object",
      "description": "",
      "default": "",
      "properties": {
        "new_param_1": {
          "type": "string"
        },
        "new_param_2": {
          "type": "string"
        }
      }
    }
  },
  "allOf": [
    {
      "$ref": "#/definitions/new_group_1"
    }
  ]
}

2. Publish Location and `params.outdir`

For each run, Foundry defines a publish location in the Run page → Advanced tab.

This field is auto-filled based on the run environment; so you do not need to set it manually on the run page.
Foundry then sets params.outdir by taking this publish location and appending the suffix: "/report{runID}".

This approach lets Foundry organize outputs consistently across runs without requiring you to manage output paths manually.

In your Nextflow pipelines, set the publish directory to params.outdir. This aligns your custom pipeline with nf-core conventions and Foundry’s expectations. Once this is done, Foundry will no longer show params.outdir as a separate input on the run form.

3. Showing Report Files in the Report Tab

To display specific report files under the Report tab in Foundry:

Add a YAML configuration file named either tower.yml or foundry.yml to your pipeline repository.
The configuration file should follow the nf-core format.

You can see an example configuration here: https://github.com/nf-core/rnaseq/blob/master/tower.yml

reports:
  multiqc_report.html:
    display: "MultiQC HTML report"
  "**/star_salmon/**/deseq2.plots.pdf":
    display: "All samples STAR Salmon DESeq2 QC PDF plots"

Any report files matching the patterns defined in tower.yml or foundry.yml will appear in Foundry’s Report section for that run.