liquibase-refactor-schema-of-java-app

Posted on: 2019-10-08 2019-11-19
Tags: Persistence

Use Liquibase to Safely Evolve Your Database Schema

1. Overview

In this quick tutorial, we’ll make use of Liquibase to evolve the database schema of a Java web application.

We’re going to focus on a general Java app first, and we’re also going to take a focused look at some interesting options available for Spring and Hibernate.

Very briefly, the core of using Liquibase is the changeLog file – an XML file that keeps track of all changes that need to run to update the DB.

Let’s start with the Maven dependency we need to add into our pom.xml:

<dependency>
    <groupId>org.liquibase</groupId>
     <artifactId>liquibase-core</artifactId>
      <version>3.4.1</version>
</dependency>

You can also check if there’s a newer version of liquibase-core here.

<databaseChangeLog
  xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
  xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog-ext
   http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd
   http://www.liquibase.org/xml/ns/dbchangelog
   http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.4.xsd">

    <changeSet author="John" id="someUniqueId">
        <addColumn tableName="users">
            <column name="address" type="varchar(255)" />
        </addColumn>
    </changeSet>

</databaseChangeLog>

Note how the change set is identified by an id and an author – to make sure it can be uniquely identified and only applied once.

Let’s not see how to wire this into our application and make sure that it runs when the application starts up.

3. Run Liquibase with a Spring Bean

Our first option to run the changes on application startup is via a Spring bean. There are of course many other ways, but if we’re dealing with a Spring application – this is a good, simple way to go:

@Bean
public SpringLiquibase liquibase() {
    SpringLiquibase liquibase = new SpringLiquibase();
    liquibase.setChangeLog("classpath:liquibase-changeLog.xml");
    liquibase.setDataSource(dataSource());
    return liquibase;
}

Note how we’re pointing it to a valid changeLog file that needs to exist on the classpath.

4. Use Liquibase with Spring Boot

If you are using Spring Boot, there is no need to define a Bean for Liquibase.

All you need is to put your change log in “db/changelog/db.changelog-master.yaml” and Liquibase migrations will run automatically on startup.

Note that:

You need to add “liquibase-core” dependency.
You can change default change log file using “liquibase.change-log” property – for example:

liquibase.change-log=classpath:liquibase-changeLog.xml

5. Disable Liquibase in Spring Boot

Sometimes, we may need to disable Liquibase migration’s execution on startup.

The simplest option we have is to use a spring.liquibase.enabled property. This way, all the remaining Liquibase configuration stays untouched.

Here’s the example for Spring Boot 2:

spring.liquibase.enabled=false

For Spring Boot 1.x, we need to use a liquibase.enabled property:

liquibase.enabled=false

**6. Generate the changeLog with a Maven Plugin**

Instead of writing the changeLog file manually – we can use the Liquibase Maven plugin to generate one and save ourselves a lot of work.

6.1. Plugin Configuration

Here is the changes to our pom.xml:

<dependency>
    <groupId>org.liquibase</groupId>
    <artifactId>liquibase-maven-plugin</artifactId>
    <version>3.4.1</version>
</dependency>
...
<plugins>
    <plugin>
        <groupId>org.liquibase</groupId>
        <artifactId>liquibase-maven-plugin</artifactId>
        <version>3.4.1</version>
        <configuration>
            <propertyFile>src/main/resources/liquibase.properties</propertyFile>
        </configuration>
    </plugin>
</plugins>

**6.2. Generate a ChangeLog from an Existing Database**

We can use the plugin to Generate a Changelog from an Existing Database:

mvn liquibase:generateChangeLog

Here are the liquibase properties:

url=jdbc:mysql://localhost:3306/oauth_reddit
username=tutorialuser
password=tutorialmy5ql
driver=com.mysql.jdbc.Driver
outputChangeLogFile=src/main/resources/liquibase-outputChangeLog.xml

The end result is an changeLog file that we can use either to create an initial DB schema or to populate data. Here’s how that would look like for our example app:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<databaseChangeLog ...>

    <changeSet author="John (generated)" id="1439225004329-1">
        <createTable tableName="APP_USER">
            <column autoIncrement="true" name="id" type="BIGINT">
                <constraints primaryKey="true"/>
            </column>
            <column name="accessToken" type="VARCHAR(255)"/>
            <column name="needCaptcha" type="BIT(1)">
                <constraints nullable="false"/>
            </column>
            <column name="password" type="VARCHAR(255)"/>
            <column name="refreshToken" type="VARCHAR(255)"/>
            <column name="tokenExpiration" type="datetime"/>
            <column name="username" type="VARCHAR(255)">
                <constraints nullable="false"/>
            </column>
            <column name="preference_id" type="BIGINT"/>
            <column name="address" type="VARCHAR(255)"/>
        </createTable>
    </changeSet>
    ...
</databaseChangeLog>

**6.3. Generate a ChangeLog from Diff Between Two Databases**

We can use the plugin to generate a changeLog file from the differences between two existing databases (for example: development and production):

mvn liquibase:diff

Here are the properties:

changeLogFile=src/main/resources/liquibase-changeLog.xml
url=jdbc:mysql://localhost:3306/oauth_reddit
username=tutorialuser
password=tutorialmy5ql
driver=com.mysql.jdbc.Driver
referenceUrl=jdbc:h2:mem:oauth_reddit
diffChangeLogFile=src/main/resources/liquibase-diff-changeLog.xml
referenceDriver=org.h2.Driver
referenceUsername=sa
referencePassword=

And here’s a snippet of the generated changeLog:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<databaseChangeLog ...>
    <changeSet author="John" id="1439227853089-1">
        <dropColumn columnName="address" tableName="APP_USER"/>
    </changeSet>
</databaseChangeLog>

This is a super powerful way to evolve your DB by – for example – allowing Hibernate to auto-generate a new schema for development, and then using that as a reference point against the old schema.

7. Use the Liquibase Hibernate Plugin

If the application uses Hibernate – we’re going to take a look at a very useful way of generating the changeLog.

First – here’s how the liquibase-hibernate plugin should be configured in Maven:

7.1. Plugin Configuration

First, let’s get the new plugin configured and using the right dependencies:

<plugins>
    <plugin>
        <groupId>org.liquibase</groupId>
        <artifactId>liquibase-maven-plugin</artifactId>
        <version>3.4.1</version>
        <configuration>
            <propertyFile>src/main/resources/liquibase.properties</propertyFile>
        </configuration>
        <dependencies>
            <dependency>
                <groupId>org.liquibase.ext</groupId>
                <artifactId>liquibase-hibernate4</artifactId>
                <version>3.5</version>
            </dependency>
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-beans</artifactId>
                <version>4.1.7.RELEASE</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.data</groupId>
                <artifactId>spring-data-jpa</artifactId>
                <version>1.7.3.RELEASE</version>
            </dependency>
        </dependencies>
    </plugin>
</plugins>

**7.2. Generate a changeLog from Diffs Between a Database and Persistence Entities**

Now, for the fun part. We can use this plugin to generate a changeLog file from the differences between an existing database (for example production) and our new persistence entities.

So – to make things simple – once an entity is modified, you can simply generate the changes against the old DB schema, getting a clean, powerful way to evolve your schema in production.

Here are the liquibase properties:

changeLogFile=classpath:liquibase-changeLog.xml
url=jdbc:mysql://localhost:3306/oauth_reddit
username=tutorialuser
password=tutorialmy5ql
driver=com.mysql.jdbc.Driver
referenceUrl=hibernate:spring:org.baeldung.persistence.model
  ?dialect=org.hibernate.dialect.MySQLDialect
diffChangeLogFile=src/main/resources/liquibase-diff-changeLog.xml

Note: The referenceUrl is using package scan, so the dialect parameter is required.

8. Conclusion

In this tutorial we illustrated several ways to use Liquibase and get to a safe and mature way of evolving and refactoring the DB schema of a Java app.

The implementation of all these examples and code snippets can be found in my github project – this is an Eclipse based project, so it should be easy to import and run as it is.

getdocs

3043