DataWeave 2.0 Syntax Changes with examples

Mule 4 was released in early 2018. One of the major change in Mule 4 is, making DataWeave a default expression language over Mule 3’s default Mule Expression Language. XML namespace of DataWeave is moved from dw to ee (core) and version has changed from 1.0 to 2.0 in Mule 4.

Apart from syntax changes, there are many new feature additions to DataWeave 2.0. In this post, we will focus on the syntax changes in 2.0 compared to 1.0.

Changes covered in this post may not be the list of all syntax changes and there may be more changes. Other new features are covered in various posts here.

Let’s begin with a random quote about change -

Your success in life isn’t based on your ability to simply change. It is based on your ability to change faster than your competition, customers, and business.
— Mark Sanborn

DataWeave script file can be divided into two sections -

  1. Header that defines directives

  2. Body that defines the output

1. DataWeave Header Changes

The DataWeave version has changed from 1.0 to 2.0. Along with that, below listing summarizes changes to the header sections of dataweave -

  1. dw version changed from %dw 1.0 to %dw 2.0

  2. All directives in DataWeave 1.0 must start with %. This is no longer needed in 2.0 (except for %dw declaration).

  3. Custom functions in DataWeave 1.0 are defined with function directive. In DataWeave 2.0, it has been changed to fun. Also, the function body and function signature must separate by = instead of a space.

In both DataWeave versions, you can define custom functions with either var or function(/fun) directives.

IMO, all these changes may be trivial but good. Removal of %, saves you some characters and makes header section look more like regular code instead of directives.

Below two code listings show the same script written in both versions -

Listing:1.A - DataWeave 1.0 Headers
%dw 1.0
%output application/json
%var user = {"firstName": "Manik", "lastName" : "Magar"}
%var foo = "bar"
%input payload application/json
%function getName(data) (data.firstName ++ ' ' ++ data.lastName)
%var toFullname = (data) -> (data.firstName ++ ' ' ++ data.lastName)
---
{
	"foo" : foo,
	"getName" : getName(user),
	"toFullname" : toFullname(user)
}
Listing:1.B - DataWeave 2.0 Headers
%dw 2.0
output application/json
input payload application/json
var user = {"firstName": "Manik", "lastName" : "Magar"}
var foo = "bar"
fun getName(data) = (data.firstName ++ ' ' ++ data.lastName)
var toFullname = (data) -> (data.firstName ++ ' ' ++ data.lastName)
---
{
	"foo" : foo,
	"getName" : getName(user),
	"toFullname" : toFullname(user)
}
Listing:1.C - Output of Both Scripts
{
  "foo": "bar",
  "getName": "Manik Magar",
  "toFullname": "Manik Magar"
}

2. DataWeave Body Changes

DataWeave script’s body section is where you write all your transformation logic that defines output structure. Some interesting changes have happened to body section of the script as well. Let’s look at those changes.

Below grouping is not an official grouping by MuleSoft but just something I came up with. Makes it easy to write and follow :).

2.1 Iterative Functions

You guessed it right! These are the functions that allow you to iterate and operate on collection of objects or single object. This can include functions like map ,mapObject, filter, pluck, groupBy etc. Some interesting changes have been made for these functions in DataWeave 2.0.

2.1.1 Automatic coercion from Object to Array

DataWeave 1.0 allowed automatic coercion of object to array. What that means, is in DataWeave 1.0, map operator which allows you to iterate over array or collection of objects, was also working if the input was an object.

Consider below DataWeave 1.0 script where we expected to receive a list of users to map. For some reason, at runtime we received a single user object instead of collection of users [why? because bugs are everywhere :)].

Our script should have either failed or generated single user object, but it generates two user objects with some really incorrect data.
Listing:2.1.1.A - DataWeave 1.0 Object to Array Coercion
%dw 1.0
%output application/json
%var users = {"firstName": "Manik", "lastName": "Magar"}
---
users map {
	"firstName": $.firstName,
	"lastName": $.lastName
}
Listing:2.1.1.B - Output
[
  {
    "firstName": "Manik",
    "lastName": null
  },
  {
    "firstName": null,
    "lastName": "Magar"
  }
]

Wonder why?

Because DataWeave 1.0 treated every property (key-value pair) of an object as single object and then our input object to be a collection of those objects, like array shown in below listing. Now, if you provide this as an input to map, you are going to get two objects with some null fields.

But was that the expected output from original script? Off course, Not.

Listing:2.1.1.C - Object Coerced to an Array
[
  {
    "firstName": "Manik"
  },
  {
    "lastName": "Magar"
  }
]

DataWeave 2.0 has generously corrected this behavior and removed the automatic coercion of objects to array. With, DataWeave 2.0, similar script will generate an error (in preview mode, as well as at runtime)

Data2.0 Automatic coercion of Objects to Array
Did you notice the error details shown in studio 7? They are so detailed and clear! Yet Another Useful Change!

IMO, This change can definitely safeguard from having unexpected output and data corruption in the downstream systems. Well done DataWeave 2.0!

2.1.2 Overloaded Functions

In DataWeave 2.0, The functions filter and groupBy are overloaded to operate on objects. This could be useful to create sub-objects from a large object.

Consider an object {"firstName": "Manik", "lastName": "Magar", "Address Line1" : "Address line 1"} and we want to create another object with all the properties where value contains 'Ma'.

With DataWeave 1.0, we will need to use mapObject and then map all key-value pairs conditionally. Something like below -

Listing:2.1.2.A - DataWeave 1.0 Creating sub-objects
%dw 1.0
%output application/json
%var user = {"firstName": "Manik", "lastName": "Magar", "Address Line1" : "Address line 1"}
---
name : user mapObject {
	('$$' : '$') when ($ contains 'Ma')
}
Listing:2.1.1.B - output
{
  "name": {
    "firstName": "Manik",
    "lastName": "Magar"
  }
}

In DataWeave 2.0, we can leverage the overloaded function of filter which allows us to operate on objects and achieve same result with shorter code.

Listing:2.1.1.C - DataWeave 2.0 Creating sub-objects with filter function
%dw 2.0
output application/json
var user = {"firstName": "Manik", "lastName": "Magar", "Address Line1" : "Address line 1"}
---
name : user filter ($ contains "Ma")

Above one-line and simple code generates exact same output as-in Listing:2.1.1.B. To understand the difference, if you write that filter function in DataWeave 1.0, the output will not be an object but an array of values satisfying given condition.

Listing:2.1.1.D - DataWeave 1.0 Filter Object Output
{
  "name": [
    "Manik",
    "Magar"
  ]
}

Similarly, groupBy function has been overloaded to allow invoking three-argument lambda on object -

Listing:2.1.1.E - DataWeave 2.0 Group By an Object with overloaded function
%dw 2.0
output application/json
var user = {"firstName": "Manik", "lastName": "Magar", "Address Line1" : "Address line 1"}
---
user groupBy ((value, key) -> if (key contains 'Name') "Names" else "Address")
Listing 2.1.1.E also gives you a sneak peak at if …​ else …​ that replaces when …​ otherwise …​ conditional syntax of DataWeave 1.0. We will see more about it in later part.

Running this script against our user object, will partition (group) the object as below -

Listing:2.1.1.F - output
{
  "other": {
    "Address Line1": "Address line 1"
  },
  "Names": {
    "firstName": "Manik",
    "lastName": "Magar"
  }
}
In DataWeave 1.0, The lambda signature is (value, index) so it is not possible to write similar group by on an object.

If you run groupBy on an object in DataWeave 1.0, it does not output keys which could be another problem.

Listing:2.1.1.G - DataWeave 1.0 Group By an Object
%dw 1.0
%output application/json
%var user = {"firstName": "Manik", "lastName": "Magar", "Address Line1" : "Address line 1"}
---
user groupBy ((value, index) -> "Name" when (value contains 'Ma') otherwise "Address")
Listing:2.1.1.H - output
{
  "Address": [
    "Address line 1"
  ],
  "Name": [
    "Manik",
    "Magar"
  ]
}

To achieve, the output like Listing:2.1.1.F in DataWeave 1.0, we will need to take approach similar to shown in Listing:2.1.2.A.

2.1.3 Addition of third parameter

DataWeave 2.0 have added index as 3rd parameter to mapObject, pluck, filter, and groupBy. Some of these functions also have an index in DataWeave 1.0 but as a second parameter.

Consider below two code listings -

Listing:2.1.3.A - DataWeave 1.0 New Parameter addition
%dw 1.0
%output application/json
%var user = {"firstName": "Manik", "lastName": "Magar", "addressLine1" : "Address line 1"}
%var userList = [{"firstName": "Manik", "lastName": "Magar", "addressLine1" : "Address line 1"},
				{"firstName": "Om", "lastName": "Magar", "addressLine1" : "Address line 2"}
				]
---
{
	mapObject: user mapObject (value, key) -> {    (1)
		'$key': value
	},

	pluck: user pluck (value, key) -> (key ++ '-' ++ value),   (2)

	filter: user filter ((value, index) -> (index < 1)),   (3)
	filter2: userList filter (value, index) -> (value.firstName contains 'Ma'),    (4)

	groupBy1 : user groupBy ((value, index) -> (index < 1)),     (5)
	groupBy2 : userList groupBy ((value, index) -> value.lastName)     (6)

}
Listing:2.1.3.B - DataWeave 2.0 New Parameter addition
%dw 2.0
output application/json
var user = {"firstName": "Manik", "lastName": "Magar", "addressLine1" : "Address line 1"}
var userList = [{"firstName": "Manik", "lastName": "Magar", "addressLine1" : "Address line 1"},
				{"firstName": "Om", "lastName": "Magar", "addressLine1" : "Address line 2"}
				]
---
{
	mapObject: user mapObject (value:String, key:Key, index:Number) -> { (1)
		'$key-$index': value
	},

	pluck: user pluck (value, key, index) -> (key ++ '-' ++ index ++ '-' ++ value),    (2)

	filter: user filterObject ((value, key, index) -> (index < 1)), (3)
	filter2: userList filter ((value, index) -> (index < 1)), (4)

	groupBy1 : user groupBy (value, key) -> key endsWith 'Name',    (5)
	groupBy2 : userList groupBy ((value, index) -> value.lastName)   (6)

}

Listing 2.1.3 A vs B:

1 Third parameter index have been added to mapObject function. This could be useful if you want to iterate on object properties based on property indices.
2 Addition of third index parameter to pluck, similar to mapObject.
3 filter function had index parameter in 1.0 too but When an object is passed to filter function, index is moved to the third place and a new second param key has been introduced to the lambda. This is a great addition to operate over objects.
4 There is NO change to filter lambda when input is a collection.
5 groupBy function on Object, no longer supports index parameter. It is replaced with getting a key as a second parameter.
6 There is NO change to gruopBy when input is a collection.

2.1.4 Allow NULL input

Things happen, payloads becomes null sometime and when that happens, script starts failing. In DataWeave 1.0, functions will error out due to null input.

In DataWeave 2.0, map, mapObject and filter functions have been modified to accept NULL input and when that happens, they just return NULL.

Consider below DataWeave 1.0 and 2.0 scripts -

Listing:2.1.4.A - DataWeave 1.0 Null input
%dw 1.0
%output application/json
---
{
	user: null filter ($.firstName == 'Manik'),

	user: null mapObject {
		'$$': $
	},

	users : null map {
		name: $.firstName
	}
}

All of the above mappings in DataWeave 1.0 will fail to compile due to NULL input.

Listing:2.1.4.B - DataWeave 1.0 Null input
%dw 2.0
output application/json
---
{
	user: null filter ($.firstName == 'Manik'),

	user: null mapObject {
		'$$': $
	},

	users : null map ((value, index) -> value.firstName)
}

All these mappings in DataWeave 2.0 will not fail but generates below output.

Whether this is acceptable output or not, depends on your use case. You should consider the null input case and handle it appropriately.
Listing:2.1.4.B - Output of DataWeave 2.0
{
  "user": null,
  "user": null,
  "users": null
}
NULL for map is allowed when lambda is used and returns a single value that can be made null i.e. (T, Number) → Boolean format. For example, consider below mapping that tries to output a key-value pair, but will error out in DataWeave 2.0 too. 
users : null map ((value, index) -> (name: value.firstName))

2.2 Operator Changes

As a language, DataWeave 1.0 defines many operators like is, upper, typeOf etc. that help in transformations. Changes have been made to the way operators are used in DataWeave 2.0.

2.2.1 Operators are now Functions

In DataWeave 2.0, all of these are made as functions. What that means is, they should be called like any other function calls and with their arguments using parenthesis.

Consider below DataWeave 1.0 and 2.0 code listings that shows syntax for operators/functions and produces same results -

Listing 2.2.1.A DataWeave 1.0 Operators
%dw 1.0
%output application/json
---
{
	aType: typeOf "Manik",
	aSize: sizeOf [1,2,3,4],
	aUpper: upper "Manik"
}
Listing 2.2.1.B DataWeave 2.0 Operators
%dw 2.0
output application/json
---
{
	aType: typeOf("Manik"),
	aSize: sizeOf([1,2,3,4]),
	aUpper: upper("Manik")
}

2.2.2 Range Selection

DataWeave 1.0 uses .. operator to select ranges. This have been replaced with a new to operator.

Consider below DataWeave 1.0 and 2.0 code listings that shows syntax for operators/functions and produces same results -

Listing 2.2.2.A DataWeave 1.0 Range Selector
%dw 1.0
%output application/json
%var data = [1,2,3,4,5,6,7,8,9]
%var m = 3
%var n = 5
---
{
	subArray: data[2..4],
	//Throws error
	subArray2: data [m..n],		(1)
	subString: "ABCDEFG" [1..3]
}
1 Using dynamic ranges with .. was so error prone and not so straightforward. This throws error in DataWeave 1.0.
Listing 2.2.2.B DataWeave 2.0 Range Selector
%dw 2.0
output application/json
var data = [0,1,2,3,4,5,6,7,8,9]
var m = 3
var n = 5
---
{
	subArray: data [2 to 4],
	subArray2: data [m to n],		(1)
	subString: "ABCDEFG" [1 to 3]

}
1 Using dynamic variables to define range is straight forward and easy!
.. is still a valid descendants selector for DataWeave 2.0.

2.2.3 Conditional Logic

DataWeave 1.0 uses when, unless (probably rarely used) and otherwise to implement conditional logic. All these keywords have been replaced by if and else in DataWeave 2.0.

Consider below DataWeave 1.0 and 2.0 code listings that shows syntax for operators/functions and produces same results -

Listing 2.2.3.A DataWeave 1.0 Conditional Logic
%dw 1.0
%output application/json
%var data = [1,2,3,4,5,6,7,8,9]
---
{
	has2: "true" when data contains 2 otherwise "false",
	has22: "true" when data contains 22 otherwise "false",
	has21: "false" unless data contains 21 otherwise "true",
	has2: "false" unless data contains 2 otherwise "true",
	has22Or5: "true" when data contains 22
					otherwise ("true5"
							when data contains 5 otherwise "false"
					)
}
Listing 2.2.3.B DataWeave 2.0 Conditional Logic
%dw 2.0
output application/json
var data = [0,1,2,3,4,5,6,7,8,9]
---
{
	has2: if (data contains 2) "true" else "false",
	has22: if (data contains 22) "true" else "false",

	//outputs true5
	has22or5: if (data contains 22) "true"		(1)
					else if (data contains 5) "true5"
					else "false"
}
1 Nested conditions if .. else if .. else feels so familiar that of in Java!

2.2.4 Binary functions

Binary functions are the functions need two arguments. In DataWeave 1.0, they have a notation of argument1 function argument2. DataWeave 2.0 have added an alternate notation to all these binary functions, making them really look like function calls.

Consider below DataWeave 2.0 script where all key1 like calls are valid for both DataWeave versions and key2 like calls are new notations added in DataWeave 2.0.

Listing 2.2.4.A DataWeave Binary Functions
%dw 2.0
output application/json
var data = [0,1,2,3,4,5,6,7,8,9]
---
{
	contains1: data contains 2,
	contains2: contains(data, 2),
	starts1: "Manik" startsWith "M",
	starts2: startsWith("Manik", "N")
}

2.3 Other Changes

Few other syntax changes have been made in DataWeave 2.0. Here is a list of those changes -

  • Removed : from all type names and they are written with first upper case letter i.e Object instead of :object.

  • Added new key-value pair selector .& that returns all matching key-values as single object. For example {"firstName": "MM1", "addr": "addr1", "addr": "addr2"}.&addr will return an object {"addr": "addr1","addr": "addr2"}.

  • Namespace prefixes can no longer contain the character -.

  • New namespace selector (.#) returns the namespace used.

  • New supported syntax for match when using arrays [head ~ tail].

3. What else?

We looked at the syntax changes in DataWeave 2.0 compared to DataWeave 1.0 and if you are thinking these are the only changes then you are missing lot of new additions.

As I mentioned earlier, DataWeave 2.0 is the default expression language in Mule 4 which itself opens multiple opportunities to leverage DataWeave power throughout your mule flow.

Apart from that, it also introduced concepts like reusable modules that can contain your common functions, operators etc. Many new functions and operators have been added, features like importing java classes and calling static methods from within DataWeave enriches overall DataWeave usability and power.

So I strongly, suggest you to go through the DataWeave documentation (reference links are at the bottom) and make yourself familiar with DataWeave 2.0.

4. Conclusion

Above section pretty much concludes this very lengthy post, so I don’t think I should say much here. Anyways, here is something - Keep learning new DataWeave and Mule 4 to enjoy the ride! Do let me know your thoughts or questions related to this!

5. Reference and Further Reading

  • Check out other posts about DataWeave here

  • Learn more about mule here

on twitter to get updates on new posts.
MEL Global Functions and their use
Part 1: Mule Batch Processing - Introduction

Stay updated!

On this blog, I post articles about different technologies like Java, MuleSoft, and much more.

You can get updates for new Posts in your email by subscribing to JavaStreets feed here -


{ Manik Magar }

Lives on Java Planet, Walks on Java Streets, Read/Writes in Java, JCP member, Jakarta EE enthusiast, MuleSoft Integration Architect, MuleSoft Community Ambassador, Open Source Contributor and Supporter, also writes at Unit Testers, A Family man!