Skip to main content
Version: Next

Batch Transform

Applies a function to each item of an array and hands the array on as one message. The items are processed in parallel.

Type: batch-transform · Category: Transform · Ports: one input · one output

The Batch Transform node as it appears on the pipeline canvas.The Batch Transform node as it appears on the pipeline canvas.

When to use it

  • Converting units, renaming fields or enriching every row of a result set
  • Any per-item work where the items do not depend on each other
Reach for something else when

If each item must travel through the rest of the graph on its own — one HTTP call per row, say — use Array Iterator instead.

Settings

SettingKeyDefaultPurpose
Transform typetransformTypemapHow items are combined back.
CodecodeThe per-item function.
PayloadpayloadType / payloadValuemsgWhere the collection comes from.

Example — converting a batch of readings

def transform(item, message):
"""
:param item: the current item from the incoming array
:param message: the parent message context
:return: the transformed item
"""
return {**item, "celsius": round((item["fahrenheit"] - 32) * 5 / 9, 2)}

In

{ "payload": [ { "sensor": "T1", "fahrenheit": 160.2 },
{ "sensor": "T2", "fahrenheit": 158.8 } ] }

Out

{ "payload": [ { "sensor": "T1", "fahrenheit": 160.2, "celsius": 71.22 },
{ "sensor": "T2", "fahrenheit": 158.8, "celsius": 70.44 } ] }

Gotchas

  • Ordering across items is not guaranteed, because they run in parallel. Never accumulate into shared state here — compute per item and combine afterwards.

See also