Skip to content

Pipeline Guide

Foundry supports both Via and nf-core/Nextflow pipelines. This section explains how to export pipelines from Foundry to a remote Git/Bitbucket/Gitlab repository.

Exporting Via and nf-core/Nextflow Pipelines

You can export both Via and nf-core/Nextflow pipelines from Foundry using the same workflow:

  1. Go to the Pipelines page and select the pipeline you want to export.
  2. Click the three-dot (⋮) menu in the top-right corner and choose Export Pipeline.
  3. Select the repository credentials you configured under Profile → Integration Tokens.
  4. Enter the Repository URL (for example, https://github.com/your-org/your-pipeline).
  5. (Optional) Enter the tag/branch you want to use (for example, main or 3.6.1).
  6. Click Export Pipeline to start the export.

For Via pipelines, Foundry:

  • Generates an extra importable .vf file representing the pipeline.
  • Pushes this .vf file to the selected repository using the credentials and repository URL you provided.

These exported .vf files can later be re-imported into Foundry using the Via pipeline import flow.

Pipelines Holding a Process Imported from a Repository

Not available in Foundry Connect 2.22.0 or earlier

This check arrives in the release that follows 2.22.0.

A process imported from a Git repository (see Importing Processes from a Repository) is stored in Foundry Connect but is not executed. A .vf file does not record where a process came from, so a process carried inside one would reach another installation looking like any other process, and could run there.

Foundry Connect therefore refuses to put a process imported from a repository into a .vf file. The refusal covers every action in the pipeline's three-dot (⋮) menu that builds one:

  • Export Pipeline, which pushes a .vf file to your repository.
  • Download Pipeline, whose zip file for a Via pipeline includes a .vf file.
  • Download VF File, shown to admins for Via pipelines.

It applies wherever the process sits in the pipeline, including inside a module the pipeline uses. Nothing is exported or downloaded, and the message names the process and the repository it came from:

Process "name" was imported from repository URL and cannot be exported into a .vf file. A .vf file does not record where its code came from, so an installation importing it could run code that nobody reviewed. Import the process from that repository on the other installation instead.

To use the process on another installation, import it there from the same repository.

Exporting Pipeline Parameters

To facilitate the exporting of used parameters in a pipeline, we have implemented a process called publish_params_as_JSON. This process can be found by using the search bar on the pipeline page. To enable this capability in a specific pipeline, you can simply drag and drop the publish_params_as_JSON process into the desired location.

Once you have added the process to the pipeline, please drag and drop the output parameter to the workflow and rename it as logs.

After the run completes, a file named run_params.json will be generated under the "logs" folder, containing the exported parameters.

[
    {
        "Adapter_Removal": {
            "min_length" : 11, 
            "Adapter_Sequence":"AGATCGGAAGAGC"
        },
        "bed": "s3://viafoundry/genome_data/mouse/mm10/refseq/genes/genes.bed", 
        "hisat2Index": "s3://viafoundry/genome_data/mouse/mm10/refseq/Hisat2Index", 
        "outdir": "s3://viafoundry/runs/report159"
    }
]