Channels

Nextflow is based on the dataflow programming model in which processes communicate through channels.

A channel has two major properties:

Sending a message is an asynchronous (i.e. non-blocking) operation, which means the sender doesn’t have to wait for the receiving process.
Receiving a message is a synchronous (i.e. blocking) operation, which means the receiving process must wait until a message has arrived.

Channel types

In Nextflow there are two kinds of channels: queue channels and value channels.

Queue channel

A queue channel is a non-blocking unidirectional FIFO queue connecting a producer process (i.e. outputting a value) to a consumer process or an operator.

A queue channel can be created by factory methods (of, fromPath, etc), operators (map, flatMap, etc), and processes (see Process outputs).

Value channel

A value channel can be bound (i.e. assigned) with one and only one value, and can be consumed any number of times by a process or an operator.

A value channel can be created with the value factory method or by any operator that produces a single value (first, collect, reduce, etc). Additionally, a process will emit value channels if it is invoked with all value channels, including simple values which are implicitly wrapped in a value channel.

For example:

process foo {
  input:
  val x

  output:
  path 'x.txt'

  script:
  """
  echo $x > x.txt
  """
}

workflow {
  result = foo(1)
  result.view { file -> "Result: ${file}" }
}

In the above example, since the foo process is invoked with a simple value instead of a channel, the input is implicitly wrapped in a value channel, and the output is also emitted as a value channel.

Channel factories

Channel factories are functions that can create channels.

For example, the Channel.of() factory can be used to create a channel from an arbitrary list of arguments:

Channel.of(1, 2, 3).view()

New in version 20.07.0: channel was introduced as an alias of Channel, allowing factory methods to be specified as channel.of() or Channel.of(), and so on.

See Channel factories for the full list of channel factories.

Operators

Channel operators, or operators for short, are functions that consume and produce channels. Because channels are asynchronous, operators are necessary to manipulate the values in a channel, aside from using a process. As a result, operators are useful for implementing the glue logic between processes.

Commonly used operators include:

combine: emit the combinations of two channels
collect: collect the values from a channel into a list
filter: select the values in a channel that satisfy a condition
flatMap: transform each value from a channel into a list and emit each list element separately
groupTuple: group the values from a channel based on a grouping key
join: join the values from two channels based on a matching key
map: transform each value from a channel with a mapping function
mix: emit the values from multiple channels
view: print each value in a channel to standard output

See Operators for the full list of operators.