Updating Nextflow syntax
This page explains how to update Nextflow scripts and config files to adhere to the Nextflow language specification, also known as the strict syntax.
Note
If you are still using DSL1, see Migrating from DSL1 to learn how to migrate your Nextflow pipelines to DSL2 before consulting this guide.
Preparing for strict syntax
The strict syntax is a subset of DSL2. While DSL2 allows any Groovy syntax, the strict syntax allows only a subset of Groovy syntax for Nextflow scripts and config files. This new specification enables more specific error reporting, ensures more consistent code, and will allow the Nextflow language to evolve independently of Groovy.
The strict syntax is currently only enforced by the Nextflow language server, which is provided as part of the Nextflow VS Code integration. However, the strict syntax will be gradually adopted by the Nextflow CLI in future releases and will eventually be the only way to write Nextflow code.
New language features will be implemented as part of the strict syntax, and not the current DSL2 parser, with few exceptions. Therefore, it is important to prepare for the strict syntax in order to use new language features in the future.
This section describes the key differences between the DSL2 and the strict syntax. In general, the amount of changes that are required depends on the amount of custom Groovy code in your scripts and config files.
Removed syntax
Import declarations
In Groovy, the import declaration can be used to import external classes:
import groovy.json.JsonSlurper
def json = new JsonSlurper().parseText(json_file.text)
In the strict syntax, use the fully qualified name to reference the class:
def json = new groovy.json.JsonSlurper().parseText(json_file.text)
Class declarations
Some users use classes in Nextflow to define helper functions or custom types. Helper functions should be defined as standalone functions in Nextflow. Custom types should be moved to the lib directory.
Note
Enums, a special type of class, are supported, but they cannot be included across modules at this time.
Note
Record types will be addressed in a future version of the Nextflow language specification.
Mixing script declarations and statements
In the strict syntax, a script may contain any of the following top-level declarations:
Feature flags
Include declarations
Parameter declarations
Workflows
Processes
Functions
Output block
Alternatively, a script may contain only statements, also known as a code snippet:
println 'Hello world!'
Code snippets are treated as an implicit entry workflow:
workflow {
println 'Hello world!'
}
Script declarations and statements cannot be mixed at the same level. All statements must reside within script declarations unless the script is a code snippet:
process foo {
// ...
}
// incorrect -- move into entry workflow
// println 'Hello world!'
// correct
workflow {
println 'Hello world!'
}
Note
Mixing statements and script declarations was necessary in DSL1 and optional in DSL2. However, it is no longer supported in the strict syntax in order to simplify the language and to ensure that top-level statements are not executed when the script is included as a module.
Assignment expressions
In Groovy, variables can be assigned in an expression:
foo(x = 1, y = 2)
In the strict syntax, assignments are allowed only as statements:
x = 1
y = 2
foo(x, y)
In Groovy, variables can be incremented and decremented in an expression:
foo(x++, y--)
In the strict syntax, use += and -= instead:
x += 1
y -= 1
foo(x, y)
For and while loops
In Groovy, loop statements, such as for and while, are supported:
for (rseqc_module in ['read_distribution', 'inner_distance', 'tin']) {
if (rseqc_modules.contains(rseqc_module))
rseqc_modules.remove(rseqc_module)
}
In the strict syntax, use higher-order functions, such as the each method, instead:
['read_distribution', 'inner_distance', 'tin'].each { rseqc_module ->
if (rseqc_modules.contains(rseqc_module))
rseqc_modules.remove(rseqc_module)
}
Lists, maps, and sets provide several functions (e.g., collect, find, findAll, inject) for iteration. See Groovy standard library for more information.
Switch statements
In Groovy, switch statements are used for pattern matching on a value:
switch (aligner) {
case 'bowtie2':
// ...
break
case 'bwamem':
// ...
break
case 'dragmap':
// ...
break
case 'snap':
// ...
break
default:
// ...
}
In the strict syntax, use if-else statements instead:
if (aligner == 'bowtie2') {
// ...
} else if (aligner == 'bwamem') {
// ...
} else if (aligner == 'dragmap') {
// ...
} else if (aligner == 'snap') {
// ...
} else {
// ...
}
Spread operator
In Groovy, the spread operator can be used to flatten a nested list:
ch.map { meta, bambai -> [meta, *bambai] }
In the strict syntax, enumerate the list elements explicitly:
// alternative 1
ch.map { meta, bambai -> [meta, bambai[0], bambai[1]] }
// alternative 2
ch.map { meta, bambai ->
def (bam, bai) = bambai
[meta, bam, bai]
}
Implicit environment variables
In Nextflow DSL1 and DSL2, environment variables can be referenced directly in strings:
println "PWD = ${PWD}"
In the strict syntax, use System.getenv() instead:
println "PWD = ${System.getenv('PWD')}"
New in version 24.11.0-edge: The env() function should be used instead of System.getenv():
println "PWD = ${env('PWD')}"
Restricted syntax
The following patterns are still supported but have been restricted. That is, some syntax variants have been removed.
Variable declarations
In Groovy, variables can be declared in many different ways:
def a = 1
final b = 2
def c = 3, d = 4
def (e, f) = [5, 6]
String str = 'foo'
def Map meta = [:]
In the strict syntax, variables must be declared with def and must not specify a type:
def a = 1
def b = 2
def (c, d) = [3, 4]
def (e, f) = [5, 6]
def str = 'foo'
def meta = [:]
Note
Because type annotations are useful for providing type checking at runtime, the language server will not report errors for Groovy-style type annotations at this time. Type annotations will be addressed in a future version of the Nextflow language specification.
Strings
Groovy supports a wide variety of strings, including multi-line strings, dynamic strings, slashy strings, multi-line dynamic slashy strings, and more.
The strict syntax supports single- and double-quoted strings, multi-line strings, and slashy strings.
Slashy strings cannot be interpolated:
def id = 'SRA001'
assert 'SRA001.fastq' ~= /${id}\.f(?:ast)?q/
Use a double-quoted string instead:
def id = 'SRA001'
assert 'SRA001.fastq' ~= "${id}\\.f(?:ast)?q"
Slashy strings cannot span multiple lines:
/
Patterns in the code,
Symbols dance to match and find,
Logic unconfined.
/
Use a multi-line string instead:
"""
Patterns in the code,
Symbols dance to match and find,
Logic unconfined.
"""
Dollar slashy strings are not supported:
$/
echo "Hello world!"
/$
Use a multi-line string instead:
"""
echo "Hello world!"
"""
Type conversions
In Groovy, there are two ways to perform type conversions or casts:
def map = (Map) readJson(json) // soft cast
def map = readJson(json) as Map // hard cast
In the strict syntax, only hard casts are supported. However, hard casts are discouraged because they can cause unexpected behavior if used improperly. Groovy-style type annotations should be used instead:
def Map map = readJson(json)
Nextflow will raise an error at runtime if the readJson() function does not return a Map.
When converting a value to a different type, it is better to use an explicit method rather than a cast. For example, to parse a string as a number:
def x = '42' as Integer
def x = '42'.toInteger() // preferred
Process env inputs and outputs
In Nextflow DSL2, the name of a process env input/output can be specified with or without quotes:
process PROC {
input:
env FOO
env 'BAR'
// ...
}
In the strict syntax, the name must be specified with quotes:
process PROC {
input:
env 'FOO'
env 'BAR'
// ...
}
Implicit process script section
In Nextflow DSL1 and DSL2, the process script: section label can almost always be omitted:
process greet {
input:
val greeting
"""
echo '${greeting}!'
"""
}
In the strict syntax, the script: label can be omitted only if there are no other sections:
process sayHello {
"""
echo 'Hello world!'
"""
}
process greet {
input:
val greeting
script:
"""
echo '${greeting}!'
"""
}
Workflow onComplete/onError handlers
Workflow handlers (i.e. workflow.onComplete and workflow.onError) can be defined in several different ways in a script, but are typically defined as top-level statements and without an equals sign:
workflow.onComplete {
println "Pipeline completed at: $workflow.complete"
println "Execution status: ${ workflow.success ? 'OK' : 'failed' }"
}
The strict syntax does not allow statements to be mixed with script declarations, so workflow handlers must be defined in the entry workflow:
workflow {
// ...
workflow.onComplete = {
println "Pipeline completed at: $workflow.complete"
println "Execution status: ${ workflow.success ? 'OK' : 'failed' }"
}
}
Note
A more concise syntax for workflow handlers will be addressed in a future version of the Nextflow language specification.
Deprecated syntax
The following patterns are deprecated. The language server reports paranoid warnings for these patterns, which are disabled by default. Enable them by selecting Nextflow > Paranoid Warnings in the extension settings. These warnings may become errors in the future.
Implicit closure parameter
In Groovy, a closure with no parameters is assumed to have a single parameter named it:
ch | map { it * 2 }
In the strict syntax, the closure parameter should be explicitly declared:
ch | map { v -> v * 2 } // correct
ch | map { it -> it * 2 } // also correct
Using params outside the entry workflow
While params can be used anywhere in the pipeline code, they are only intended to be used in the entry workflow.
In the strict syntax, processes and workflows should receive params as explicit inputs:
process foo {
input:
val foo_args
// ...
}
workflow bar {
take:
bar_args
// ...
}
workflow {
foo(params.foo_args)
bar(params.bar_args)
}
Process each input
The each process input is deprecated. Use the combine or cross operator to explicitly repeat over inputs in the calling workflow.
Process when section
The process when section is deprecated. Use conditional logic, such as an if statement or the filter operator, to control the process invocation in the calling workflow.
Process shell section
The process shell section is deprecated. Use the script block instead. The VS Code extension provides syntax highlighting and error checking to help distinguish between Nextflow variables and Bash variables.
Configuration syntax
See Configuration for a comprehensive description of the configuration language.
Currently, Nextflow parses config files as Groovy scripts, allowing the use of scripting constructs like variables, helper functions, try-catch blocks, and conditional logic for dynamic configuration:
def getHostname() {
// ...
}
def hostname = getHostname()
if (hostname == 'small') {
params.max_memory = 32.GB
params.max_cpus = 8
}
else if (hostname == 'large') {
params.max_memory = 128.GB
params.max_cpus = 32
}
The strict config syntax does not support functions, and only allows statements (e.g., variables and if statements) within closures. The same dynamic configuration can be achieved by using a dynamic include:
includeConfig ({
def hostname = // ...
if (hostname == 'small')
return 'small.config'
else if (hostname == 'large')
return 'large.config'
else
return '/dev/null'
}())
The include source is a closure that is immediately invoked. It includes a different config file based on the return value of the closure. Including /dev/null is equivalent to including nothing.
Each conditional configuration is defined in a separate config file:
// small.config
params.max_memory = 32.GB
params.max_cpus = 8
// large.config
params.max_memory = 128.GB
params.max_cpus = 32
Preserving Groovy code
There are two ways to preserve Groovy code:
Move the code to the
libdirectoryCreate a plugin
Any Groovy code can be moved into the lib directory, which supports the full Groovy language. This approach is useful for temporarily preserving some Groovy code until it can be updated later and incorporated into a Nextflow script. See <lib-directory> documentation for more information.
For Groovy code that is complicated or if it depends on third-party libraries, it may be better to create a plugin. Plugins can define custom functions that can be included by Nextflow scripts like a module. Furthermore, plugins can be easily re-used across different pipelines. See Plugins for more information on how to develop plugins.