Gavin Flood

Ramblings of an Irish software developer

Loading JSON Data Into Your Spring Application on Startup

I recently ran into a common issue where I needed some data to be loaded into my Spring application on startup. I decided to store the data in JSON files, since they are easy to configure and easy to parse. Spring Data actually provides a Jackson2RepositoryPopulatorFactoryBean class to handle this exact scenario, but I had requirement that this does not handle. Once the data was loaded into the application, any changes to it should only happen through the application. If I added new data to the JSON files then that should still be added, but changes to existing data was a no-no.

So to do this, here's what I did:

Create a new Populator

Spring Data provides ResourceReaderRepositoryPopulator to persist data read from a resource to the specified repositories. I extended this for two reasons:

  1. To use generics to specify that the entities to be persisted from the JSON data will extend IdentifiableEntity (which is just a supertype that all entities in my application extend).
  2. To add a check before each entity is persisted, preventing it from occurring if the repository already contains a matching entity.
public class AdditiveResourceReaderRepositoryPopulator<T extends IdentifiableEntity>
        extends ResourceReaderRepositoryPopulator {

    private static final Logger LOGGER = LoggerFactory.getLogger(AdditiveResourceReaderRepositoryPopulator.class);

    private final ResourceReader reader;
    private final ClassLoader classLoader;

    private ApplicationEventPublisher publisher;
    private Collection<Resource> resources;

    AdditiveResourceReaderRepositoryPopulator(ResourceReader reader) {
        this(reader, null);
    }

    private AdditiveResourceReaderRepositoryPopulator(ResourceReader reader, ClassLoader classLoader) {
        super(reader, classLoader);
        Assert.notNull(reader);

        this.reader = reader;
        this.classLoader = classLoader;
    }

    @Override
    public void setResources(Resource... resources) {
        super.setResources(resources);
        this.resources = Arrays.asList(resources);
    }

    private Object readObjectFrom(Resource resource) {
        try {
            return reader.readFrom(resource, classLoader);
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

    @Override
    public void setApplicationEventPublisher(ApplicationEventPublisher publisher) {
        this.publisher = publisher;
    }

    @Override
    @SuppressWarnings("unchecked")
    public void populate(Repositories repositories) {
        RepositoryInvokerFactory invokerFactory = new DefaultRepositoryInvokerFactory(repositories);

        for (Resource resource : resources) {
            LOGGER.info(String.format("Reading resource: %s", resource));
            Object result = readObjectFrom(resource);

            if (result instanceof Collection) {
                for (T element : (Collection<T>) result) {
                    if (element != null) {
                        persist(element, invokerFactory);
                    } else {
                        LOGGER.info("Skipping null element found in unmarshal result!");
                    }
                }
            } else {
                persist((T) result, invokerFactory);
            }
        }

        if (publisher != null) {
            publisher.publishEvent(new RepositoriesPopulatedEvent(this, repositories));
        }
    }

    @SuppressWarnings("unchecked")
    private void persist(T object, RepositoryInvokerFactory invokerFactory) {
        RepositoryInvoker invoker = invokerFactory.getInvokerFor(object.getClass());
        LOGGER.debug(String.format("Persisting %s using repository %s", object, invoker));
        if (invoker.invokeFindOne(object.getId()) == null) {
            invoker.invokeSave(object);
        }
    }

}

Create a new PopulatorFactoryBean

I extended the earlier mentioned PopulatorFactoryBean. The reason for doing this was to specify the AdditiveResourceReaderRepositoryPopulator just created as the populator for the data.

public class JsonRepositoryPopulatorFactoryBean extends Jackson2RepositoryPopulatorFactoryBean {

    private Resource[] resources;
    private ApplicationContext context;
    private RepositoryPopulator populator;
    private ObjectMapper mapper;

    @Override
    public void setResources(Resource[] resources) {
        super.setResources(resources);
        this.resources = resources;
    }

    @Override
    public void setApplicationContext(ApplicationContext applicationContext) {
        super.setApplicationContext(applicationContext);
        this.context = applicationContext;
    }

    @Override
    public void setMapper(ObjectMapper mapper) {
        this.mapper = mapper;
    }

    @Override
    protected ResourceReader getResourceReader() {
        return new Jackson2ResourceReader(mapper);
    }

    @Override
    protected ResourceReaderRepositoryPopulator createInstance() {
        ResourceReaderRepositoryPopulator initializer
                = new AdditiveResourceReaderRepositoryPopulator(getResourceReader());
        initializer.setResources(resources);
        initializer.setApplicationEventPublisher(context);

        this.populator = initializer;

        return initializer;
    }

    @Override
    public void onApplicationEvent(ContextRefreshedEvent event) {

        if (event.getApplicationContext().equals(context)) {
            Repositories repositories = new Repositories(event.getApplicationContext());
            populator.populate(repositories);
        }
    }

}

Declare a bean for the PopulatorFactoryBean

All you need to do now is declare the new PopulatorFactoryBean as a bean in your application, specify the resources to import, and you should be good to go. I did this in my data config class, but you can do this wherever suits you.

@Value("classpath:/db/users.json")
private Resource userData;

@Value("classpath:/db/permissions.json")
private Resource permissionData;

@Value("classpath:/db/roles.json")
private Resource roleData;
@Bean
public AbstractRepositoryPopulatorFactoryBean repositoryPopulatorFactory() {
    JsonRepositoryPopulatorFactoryBean factory = new JsonRepositoryPopulatorFactoryBean();
    Resource[] resources = {
            permissionData,
            roleData,
            userData
    };

    factory.setResources(resources);

    return factory;
}

That's all folks!

There you have it. Now you can load data into your applications using JSON files, add to it over time, and ensure data previously loaded isn't overwritten. Make sure to consult the Spring Data documentation if you're struggling with formatting your JSON data.