Part 2: Mule Batch Processing - MUnit Testing

In Part 1, we looked at the Batch processing capability of Mule. Just like any other software code, you should unit test mule batch jobs too. In this Part 2, we will see how we can add unit tests for our batch.

For quick reference, here are links to all parts in this series:

Part 1: Mule Batch Processing - Introduction

Part 2: Mule Batch Processing - MUnit Testing

Part 3: Mule Batch Processing - MUnit Testing (The Conclusion)

As always, here is a random quote -

I find the writing unit tests actually increases my programming speed!
— Martin Fowler

1. Introduction to MUnit

MUnit is a testing framework for Mule Applications. It is loaded with integration and unit test capabilities that can help you to improve your mule code quality. It also provides mocking capability to easily mock the external systems and behaviors.

If you are not familiar with MUnit, you may read Mule documentation to know more about it.

2. Challenges for Batch Testing

In previous post, we saw how batch jobs work in Mule and also learned various components and phases in Mule Batch. While other non-batch connectors, components in Mule may be easy to test, Mule Batch components have some challenges when it comes to unit test. There may be more or less but some of those are listed here -

  1. Batch job does not return the Batch Result to calling flow, making it difficult to test the output.

  2. By default, Mule Batch Executes asynchronously. We need to consider this when we plan to call it in our test case, as test case would expect it to be synchronous.

  3. When spying on the processors in the on:complete, assertion failures does not result in test failure. [I am not sure why this behavior is there, might open a bug with Mule]

  4. Batch steps are scopes like ForEach, Transactional etc, and so cannot be mocked or verified for call.

  5. Record and Record Variables are valid only in Batch Context, so mocking them is not straight forward.

3. Sample Batch Job

Let’s consider the simple batch job that we created in previous post and see if we can get around all of these. For reference, here is the batch code -

Listing 3.A: Sample Mule Batch Job
<batch:job name="mule-simple-batch">
        <batch:input>
            <dw:transform-message doc:name="Transform Message">
                <dw:set-payload><![CDATA[%dw 1.0
%output application/java
---
payload filter $.status != 'MessedUp']]></dw:set-payload>
            </dw:transform-message>
        </batch:input>

        <batch:process-records>
            <batch:step name="Batch_Step_1">
                <flow-ref name="batch-step-1-process" doc:name="Flow batch-step-1-process"/>
            </batch:step>
            <batch:step name="Batch_Step_2"  accept-expression="#[payload.status == 'Processing']">
            	<flow-ref name="batch-step-2-process" doc:name="Flow batch-step-2-process"/>
            </batch:step>
            <batch:step name="Batch_Step_3" accept-expression="#[payload.status == 'Not-Processing']">
            	<flow-ref name="batch-step-3-process" doc:name="Flow batch-step-3-process"/>
            </batch:step>
        </batch:process-records>

        <batch:on-complete>
            <logger message="#['Batch Processing Result: Loaded:'+ payload.loadedRecords + ', successful: '+ payload.successfulRecords + ', failed: '+ payload.failedRecords]" level="INFO" doc:name="EndLogger"/>
        </batch:on-complete>
    </batch:job>

4. Testing Input phase

Let’s provide a payload where one record is in MessedUp status and thus, When batch completes, it should have loaded only 1 record. In our very first test, we will try to validate our Input phase logic in batch execution.

Listing 4.A: MUnit Test for Input Phase
<munit:test name="batch-test-input-phase"
		description="Verifies that MessedUp records are not loaded in input phase.">

		<mock:spy messageProcessor="mule:logger" doc:name="Spy For On complete"> (1)
			<mock:with-attributes>
				<mock:with-attribute name="doc:name" whereValue="#['EndLogger']" />
			</mock:with-attributes>
			<mock:assertions-before-call>
				<munit:assert-on-equals message="#['Number of loaded records must be 1']"
					expectedValue="#[1L]" actualValue="#[payload.loadedRecords]"
					doc:name="Assert Equals" />  (2)
			</mock:assertions-before-call>
		</mock:spy>

		<dw:transform-message doc:name="Transform Message">
			<dw:set-payload><![CDATA[%dw 1.0
%output application/java
---
[
	{
		"id": 1,
		"status": 'Ready',
		"type": 'CoffeeOrder'
	},{
		"id": -1,
		"status": 'MessedUp',
		"type": 'ABCDERTREGDFG'
	}
]]]></dw:set-payload>
		</dw:transform-message>

		<synchronize:run-and-wait timeout="1000000"
			doc:name="Synchronize">  (3)
			<batch:execute name="mule-simple-batch" doc:name="Run Batch mule-simple-batch" />
		</synchronize:run-and-wait>

		<mock:verify-call messageProcessor="mule:logger"
			doc:name="Verify End Logger Call">   (4)
			<mock:with-attributes>
				<mock:with-attribute name="doc:name" whereValue="#['EndLogger']" />
			</mock:with-attributes>
		</mock:verify-call>
	</munit:test>
1 Challenge#2, Batch result is not available outside batch. So we spy on a logger component from On:Complete phase to verify number of loaded records.
2 Numerical values in assertions should be of Long datatype i.e. [1L] instead of [1], otherwise assertions will fail.
3 Challenge#1, Batch jobs are asynchronous so we use synchronize:run-and-wait scope to make them synchronous and munit to wait until batch is finished.
4 Challenge#3, Test does not fail due to assertion in spy. To ensure the batch completed until end, we verify that on:complete logger is called during execution. If spy causes assertion, then this logger will never get called, resulting in test failure.
Due to asynchronous nature of batch, synchronize:run-and-wait is the only way to run batch job in MUnit test case!

5. Testing On:Complete phase

Similar to the input phase testing, next two assert statements shows how we can spy on a logger in on:complete phase to verify processing statistics.

Listing 5.A: MUnit Test - Verify success record count
  <mock:spy messageProcessor="mule:logger" doc:name="Spy For On complete">
    <mock:with-attributes>
      <mock:with-attribute name="doc:name" whereValue="#['EndLogger']" />
    </mock:with-attributes>
    <mock:assertions-before-call>
      <munit:assert-on-equals
        message="#['Number of successful records must be 2']"
        expectedValue="#[2L]" actualValue="#[payload.successfulRecords]"
        doc:name="Assert Equals" />
    </mock:assertions-before-call>
  </mock:spy>

If we have to assert the failed record counts, then we can use below assertion -

Listing 5.B: MUnit Test - Verify failed record count
  <mock:spy messageProcessor="mule:logger" doc:name="Spy For On complete">
    <mock:with-attributes>
      <mock:with-attribute name="doc:name" whereValue="#['EndLogger']" />
    </mock:with-attributes>
    <mock:assertions-before-call>
      <munit:assert-on-equals expectedValue="#[1L]"
        actualValue="#[payload.failedRecords]" doc:name="Assert Equals" />
    </mock:assertions-before-call>
  </mock:spy>

6. Conclusion

In this part, we learned about some challenges in writing MUnit test cases for our batch jobs and then we saw how to get around some of those.

7. Next: Part 3

In next part, we will learn how to verify the message routing during various batch steps. Also, we will see how to test flows involving record variables and how we can get around challenge 4 and 5, so stay tuned.

8. References and Further Reading

on twitter to get updates on new posts.
Part 1: Mule Batch Processing - Introduction
Part 3: Mule Batch Processing - MUnit Testing (The Conclusion)

Stay updated!

On this blog, I post articles about different technologies like Java, MuleSoft, and much more.

You can get updates for new Posts in your email by subscribing to JavaStreets feed here -


{ Manik Magar }

Lives on Java Planet, Walks on Java Streets, Read/Writes in Java, JCP member, Jakarta EE enthusiast, MuleSoft Integration Architect, MuleSoft Community Ambassador, Open Source Contributor and Supporter, also writes at Unit Testers, A Family man!