smooks

Posted on: 2019-10-08 2019-11-19
Tags: Data

Introduction to Smooks

1. Overview

In this tutorial, we’ll introduce the Smooks framework.

We’ll describe what it’s, list its key features, and eventually learn how to use some of its more advanced functionality.

First of all, let’s briefly explain what the framework is meant to achieve.

2. Smooks

Smooks is a framework for data processing applications – dealing with structured data such as XML or CSV.

It provides both APIs and a configuration model that allow us to define transformations between predefined formats (for example XML to CSV, XML to JSON and more).

We can also use a number of tools to set up our mapping – including FreeMarker or Groovy scripts.

Besides transformations, Smooks also delivers other features like message validations or data splitting.

2.1. Key Features

Let’s take a look at Smooks’ main use cases:

Message conversion – transformation of data from various source formats to various output formats
Message enrichment – filling out the message with additional data, which comes from external data source like database
Data splitting – processing big files (GBs) and splitting them into smaller ones
Java binding – constructing and populating Java objects from messages
Message validation – performing validations like regex, or even creating your own validation rules

3. Initial Configuration

Let’s start with the Maven dependency we need to add to our pom.xml:

<dependency>
    <groupId>org.milyn</groupId>
    <artifactId>milyn-smooks-all</artifactId>
    <version>1.7.0</version>
</dependency>

The latest version can be found on Maven Central.

4. Java Binding

Let’s now start by focusing on binding messages to Java classes. We’ll go through a simple XML to Java conversion here.

4.1. Basic Concepts

We’ll start with a simple example. Consider the following XML:

<order creation-date="2018-01-14">
    <order-number>771</order-number>
    <order-status>IN_PROGRESS</order-status>
</order>

In order to accomplish this task with Smooks, we have to do two things: prepare the POJOs and the Smooks configuration.

Let’s see what our model looks like:

public class Order {

    private Date creationDate;
    private Long number;
    private Status status;
    // ...
}

public enum Status {
    NEW, IN_PROGRESS, FINISHED
}

Now, let’s move on to Smooks mappings.

Basically, the mappings are an XML file which contains transformation logic. In this article, we’ll use three different types of rules:

bean – defines the mapping of a concrete structured section to Java class
value – defines the mapping for the particular property of the bean. Can contain more advanced logic like decoders, which are used to map values to some data types (like date or decimal format)
wiring – allows us to wire a bean to other beans (for example Supplier bean will be wired to Order bean)

Let’s take a look at the mappings we’ll use in our case here:

<?xml version="1.0"?>
<smooks-resource-list
  xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
  xmlns:jb="http://www.milyn.org/xsd/smooks/javabean-1.2.xsd">

    <jb:bean beanId="order"
      class="com.baeldung.smooks.model.Order" createOnElement="order">
        <jb:value property="number" data="order/order-number" />
        <jb:value property="status" data="order/order-status" />
        <jb:value property="creationDate"
          data="order/@creation-date" decoder="Date">
            <jb:decodeParam name="format">yyyy-MM-dd</jb:decodeParam>
        </jb:value>
    </jb:bean>
</smooks-resource-list>

Now, with the configuration ready, let’s try to test if our POJO is constructed correctly.

First, we need to construct a Smooks object and pass input XML as a stream:

public Order converOrderXMLToOrderObject(String path)
  throws IOException, SAXException {

    Smooks smooks = new Smooks(
      this.class.getResourceAsStream("/smooks-mapping.xml"));
    try {
        JavaResult javaResult = new JavaResult();
        smooks.filterSource(new StreamSource(this.class
          .getResourceAsStream(path)), javaResult);
        return (Order) javaResult.getBean("order");
    } finally {
        smooks.close();
    }
}

And finally, assert if the configuration is done properly:

@Test
public void givenOrderXML_whenConvert_thenPOJOsConstructedCorrectly() throws Exception {
    XMLToJavaConverter xmlToJavaOrderConverter = new XMLToJavaConverter();
    Order order = xmlToJavaOrderConverter
      .converOrderXMLToOrderObject("/order.xml");

    assertThat(order.getNumber(), is(771L));
    assertThat(order.getStatus(), is(Status.IN_PROGRESS));
    assertThat(
      order.getCreationDate(),
      is(new SimpleDateFormat("yyyy-MM-dd").parse("2018-01-14"));
}

4.2. Advanced Binding – Referencing Other Beans and Lists

Let’s extend our previous example with supplier and order-items tags:

<order creation-date="2018-01-14">
    <order-number>771</order-number>
    <order-status>IN_PROGRESS</order-status>
    <supplier>
        <name>Company X</name>
        <phone>1234567</phone>
    </supplier>
    <order-items>
        <item>
            <quanitiy>1</quanitiy>
            <code>PX1234</code>
            <price>9.99</price>
        </item>
        <item>
            <quanitiy>1</quanitiy>
            <code>RX990</code>
            <price>120.32</price>
        </item>
    </order-items>
</order>

And now let’s update our model:

public class Order {
    // ..
    private Supplier supplier;
    private List<Item> items;
    // ...
}

public class Item {

    private String code;
    private Double price;
    private Integer quantity;
    // ...
}

public class Supplier {

    private String name;
    private String phoneNumber;
    // ...
}

We also have to extend the configuration mapping with the supplier and item bean definitions.

Notice that we’ve also defined separated items bean, which will hold all item elements in ArrayList.

Finally, we will use Smooks wiring attribute, to bundle it all together.

Take a look at how mappings will look like in this case:

<?xml version="1.0"?>
<smooks-resource-list
  xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
  xmlns:jb="http://www.milyn.org/xsd/smooks/javabean-1.2.xsd">

    <jb:bean beanId="order"
      class="com.baeldung.smooks.model.Order" createOnElement="order">
        <jb:value property="number" data="order/order-number" />
        <jb:value property="status" data="order/order-status" />
        <jb:value property="creationDate"
          data="order/@creation-date" decoder="Date">
            <jb:decodeParam name="format">yyyy-MM-dd</jb:decodeParam>
        </jb:value>
        <jb:wiring property="supplier" beanIdRef="supplier" />
        <jb:wiring property="items" beanIdRef="items" />
    </jb:bean>

    <jb:bean beanId="supplier"
      class="com.baeldung.smooks.model.Supplier" createOnElement="supplier">
        <jb:value property="name" data="name" />
        <jb:value property="phoneNumber" data="phone" />
    </jb:bean>

    <jb:bean beanId="items"
      class="java.util.ArrayList" createOnElement="order">
        <jb:wiring beanIdRef="item" />
    </jb:bean>
    <jb:bean beanId="item"
      class="com.baeldung.smooks.model.Item" createOnElement="item">
        <jb:value property="code" data="item/code" />
        <jb:value property="price" decoder="Double" data="item/price" />
        <jb:value property="quantity" decoder="Integer" data="item/quantity" />
    </jb:bean>

</smooks-resource-list>

Finally, we’ll add a few assertions to our previous test:

assertThat(
  order.getSupplier(),
  is(new Supplier("Company X", "1234567")));
assertThat(order.getItems(), containsInAnyOrder(
  new Item("PX1234", 9.99,1),
  new Item("RX990", 120.32,1)));

5. Messages Validation

Smooks comes with validation mechanism based on rules. Let’s take a look at how they are used.

Definition of the rules is stored in the configuration file, nested in the ruleBases tag, which can contain many ruleBase elements.

Each ruleBase element must have the following properties:

name – unique name, used just for reference
src – path to the rule source file
provider – fully qualified class name, which implements RuleProvider interface

Smooks comes with two providers out of the box: RegexProvider and MVELProvider.

The first one is used to validate individual fields in regex-like style.

The second one is used to perform more complicated validation in the global scope of the document. Let’s see them in action.

5.1. RegexProvider

Let’s use RegexProvider to validate two things: the format of the customer name, and phone number. RegexProvider as a source requires a Java properties file, which should contain regex validation in key-value fashion.

In order to meet our requirements, we’ll use the following setup:

supplierName=[A-Za-z0-9]*
supplierPhone=^[0-9\\-\\+]{9,15}$

5.2. MVELProvider

We’ll use MVELProvider to validate if the total price for each order-item is less then 200. As a source, we’ll prepare a CSV file with two columns: rule name and MVEL expression.

In order to check if the price is correct, we need the following entry:

"max_total","orderItem.quantity * orderItem.price < 200.00"

5.3. Validation Configuration

Once we’ve prepared the source files for ruleBases, we’ll move on to implementing concrete validations.

A validation is another tag in Smooks configuration, which contains the following attributes:

executeOn – path to the validated element
name – reference to the ruleBase
onFail – specifies what action will be taken when validation fails

Let’s apply validation rules to our Smooks configuration file and check how it looks like (note that if we want to use the MVELProvider, we’re forced to use Java binding, so that’s why we’ve imported previous Smooks configuration):

<?xml version="1.0"?>
<smooks-resource-list
  xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
  xmlns:rules="http://www.milyn.org/xsd/smooks/rules-1.0.xsd"
  xmlns:validation="http://www.milyn.org/xsd/smooks/validation-1.0.xsd">

    <import file="smooks-mapping.xml" />

    <rules:ruleBases>
        <rules:ruleBase
          name="supplierValidation"
          src="supplier.properties"
          provider="org.milyn.rules.regex.RegexProvider"/>
        <rules:ruleBase
          name="itemsValidation"
          src="item-rules.csv"
          provider="org.milyn.rules.mvel.MVELProvider"/>
    </rules:ruleBases>

    <validation:rule
      executeOn="supplier/name"
      name="supplierValidation.supplierName" onFail="ERROR"/>
    <validation:rule
      executeOn="supplier/phone"
      name="supplierValidation.supplierPhone" onFail="ERROR"/>
    <validation:rule
      executeOn="order-items/item"
      name="itemsValidation.max_total" onFail="ERROR"/>

</smooks-resource-list>

Now, with the configuration ready, let’s try to test if validation will fail on supplier’s phone number.

Again, we have to construct Smooks object and pass input XML as a stream:

public ValidationResult validate(String path)
  throws IOException, SAXException {
    Smooks smooks = new Smooks(OrderValidator.class
      .getResourceAsStream("/smooks/smooks-validation.xml"));
    try {
        StringResult xmlResult = new StringResult();
        JavaResult javaResult = new JavaResult();
        ValidationResult validationResult = new ValidationResult();
        smooks.filterSource(new StreamSource(OrderValidator.class
          .getResourceAsStream(path)), xmlResult, javaResult, validationResult);
        return validationResult;
    } finally {
        smooks.close();
    }
}

And finally assert, if validation error occurred:

@Test
public void givenIncorrectOrderXML_whenValidate_thenExpectValidationErrors() throws Exception {
    OrderValidator orderValidator = new OrderValidator();
    ValidationResult validationResult = orderValidator
      .validate("/smooks/order.xml");

    assertThat(validationResult.getErrors(), hasSize(1));
    assertThat(
      validationResult.getErrors().get(0).getFailRuleResult().getRuleName(),
      is("supplierPhone"));
}

6. Message Conversion

The next thing we want to do is convert the message from one format to another.

In Smooks, this technique is also called templating and it supports:

FreeMarker (preferred option)
XSL
String template

In our example, we’ll use the FreeMarker engine to convert XML message to something very similar to EDIFACT, and even prepare a template for the email message based on XML order.

Let’s see how to prepare a template for EDIFACT:

UNA:+.? '
UNH+${order.number}+${order.status}+${order.creationDate?date}'
CTA+${supplier.name}+${supplier.phoneNumber}'
<#list items as item>
LIN+${item.quantity}+${item.code}+${item.price}'
</#list>

And for the email message:

Hi,
Order number #${order.number} created on ${order.creationDate?date} is currently in ${order.status} status.
Consider contacting the supplier "${supplier.name}" with phone number: "${supplier.phoneNumber}".
Order items:
<#list items as item>
${item.quantity} X ${item.code} (total price ${item.price * item.quantity})
</#list>

The Smooks configuration is very basic this time (just remember to import the previous configuration in order to import Java binding settings):

<?xml version="1.0"?>
<smooks-resource-list
  xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
  xmlns:ftl="http://www.milyn.org/xsd/smooks/freemarker-1.1.xsd">

    <import file="smooks-validation.xml" />

    <ftl:freemarker applyOnElement="#document">
        <ftl:template>/path/to/template.ftl</ftl:template>
    </ftl:freemarker>

</smooks-resource-list>

This time we need to just pass a StringResult to Smooks engine:

Smooks smooks = new Smooks(config);
StringResult stringResult = new StringResult();
smooks.filterSource(new StreamSource(OrderConverter.class
  .getResourceAsStream(path)), stringResult);
return stringResult.toString();

And we can, of course, test it:

@Test
public void givenOrderXML_whenApplyEDITemplate_thenConvertedToEDIFACT()
  throws Exception {
    OrderConverter orderConverter = new OrderConverter();
    String edifact = orderConverter.convertOrderXMLtoEDIFACT(
      "/smooks/order.xml");

   assertThat(edifact,is(EDIFACT_MESSAGE));
}

7. Conclusion

In this tutorial, we focused on how to convert messages to different formats, or transform them into Java objects using Smooks. We also saw how to perform validations based on regex or business logic rules.

As always, all the code used here can be found over on GitHub.

getdocs

3043