Introduction

The Key-Value store API provides a minimum set of operations that are needed by the Blueriq Runtime in order to interact with a generic key-value store. It also enhances the Blueriq Runtime to be able to run in a clustered configuration.

A default implementation of the Key-Value Store API based on Redis is provided by Blueriq, named Redis Key-Value Store Component.

This chapter describes how to enable the default component and how to add a custom implementation for a key value store.

Redis Key-Value Store Component

Prerequisites

An instance of Redis server needs to run on a node and needs to be accessible from the Blueriq Runtime.

Depending on the linux distribution, a Redis package may be available from the repository in any case consult their official documentation here.

Redis does not officially support Windows. However, the Microsoft Open Tech group develops and maintains this Windows port targeting Win64. More information here.

Enable/Disable

In order to enable the component, the profile "keyvalue-redis-store" profile must be added in the bootstrap.properties

spring.profiles.active=native,keyvalue-redis-store

Properties

The Redis Key-Value Store Component defines the following properties used for 10. Concurrency Control on Multiple Nodes [editor]:

Property	Description	Required	Default Value
blueriq.keyvalue-redis-store.subscription-pool.type	The thread pool type to use for the keyspace notifications subscription thread pool.	FALSE	CACHED
blueriq.keyvalue-redis-store.subscription-pool.thread-name-prefix	The thread name prefix used for threads in the keyspace notifications subscription thread pool.	FALSE	keyspace-subscription-
blueriq.keyvalue-redis-store.subscription-pool.thread-count	Indicates how many threads should be created when using FIXED thread pools.	FALSE	0
blueriq.keyvalue-redis-store.task-pool.type	The thread pool type to use for the keyspace notifications task thread pool.	FALSE	FIXED
blueriq.keyvalue-redis-store.task-pool.thread-name-prefix	The thread name prefix used for threads in the keyspace notifications subscription thread pool.	FALSE	keyspace-task-
blueriq.keyvalue-redis-store.task-pool.thread-count	Indicates how many threads should be created when using FIXED thread pools.	FALSE	4

The Redis Key-Value Store Component uses the following default Spring Boot properties in order to connect to Redis:

Property	Description	Required	Default Value
spring.redis.host	The DNS name or IP address of the Redis server	TRUE
spring.redis.port	The port on which to connect to Redis	FALSE	6379
spring.redis.password	The password used to connect to Redis. Can be left empty if no password is required.	FALSE

The following example configuration connects the Runtime to a Redis instance running on localhost on the default port and using a password:

spring.redis.host=localhost
spring.redis.password=example

Using the key-value store in custom plug-ins

Blueriq provides an IKeyValueStore interface which can be used to interact with a generic key-value store. In order to use this interface, add blueriq-component-api to your project's dependencies:

<dependency>
  <groupId>com.blueriq</groupId>
  <artifactId>blueriq-component-api</artifactId>
  <version>${blueriq.version}</version>
</dependency>

Then, inject the IKeyValueStore in your components:

@Component
public class ExampleComponent {
 
  private final IKeyValueStore keyValueStore;
 
  public ExampleComponent(IKeyValueStore keyValueStore) {
    this.keyValueStore = keyValueStore;
  }
 
  public void operation() {
    keyValueStore.set(namespacedKey("key"), "value");
  }
 
  private String namespacedKey(String key) {
    return "example" + keyValueStore.getNamespaceSeparator() + key;
  }
}

Note that we recommend using namespaces for your keys, in order to keep them separate from keys used by Blueriq or Spring Session. The root namespace used by Blueriq is "blueriq". The root namespace used by Spring Session is "spring".

If you would like to hide the namespace logic, the NamespacedKeyValueStore implementation may be used, available in the blueriq-runtime artifact. The code example below writes the same "example:key" key in the key-value store, but without having to explicitly prefix the key with a namespace every time:

@Component
public class ExampleComponent {
 
  private final IKeyValueStore keyValueStore;
 
  public ExampleComponent(IKeyValueStore keyValueStore) {
    this.keyValueStore = new NamespacedKeyValueStore(keyValueStore, "example");
  }
 
  public void operation() {
    keyValueStore.set("key", "value");
  }
}

Using other key-value store implementations

It is possible to replace the default Redis-based key-value store implementation with an implementation that uses another type of key-value store. This section shows as an example how to use Hazelcast as a key-value store and session store.

Create project and add dependencies

Create a maven project and add hazelcast and blueriq-component-api to your project's dependencies.

example pom.xml

<project ...>
  ...
  <dependencies>
    <dependency>
      <groupId>com.hazelcast</groupId>
      <artifactId>hazelcast-all</artifactId>
      <version>...</version>
    </dependency>
    <dependency>
      <groupId>com.blueriq</groupId>
      <artifactId>blueriq-component-api</artifactId>
      <version>...</version>
    </dependency>    
  </dependencies>
</project>

Implement IKeyValueStore

Implement the IKeyValueStore interface. The following example skims over the details but highlights some important issues:

import com.blueriq.component.api.store.keyvalue.IKeyValueStore;
// other imports
 
public class HazelcastKeyValueStore implements IKeyValueStore {
 
  private IMap<String, Serializable> hazelcastMap;
  public KeyValueHazelcastStore(IMap<String, Serializable> hazelcastMap) {
    this.hazelcastMap = hazelcastMap;
  }

  @Override
  public void set(String key, Serializable value) {
    notNull(key, "Key is required");

    try {
      hazelcastMap.set(key, value);
    } catch (Exception ex) {
      throw new KeyValueStoreException(ex);
    }
  }

  @Override
  public <T extends Serializable> T get(String key, Class<T> valueType) {
    notNull(key, "Key is required");
    notNull(valueType, "Value type is required");

    try {
      Serializable storedValue = hazelcastMap.get(key);

      if (storedValue == null) {
        return null;
      }

      if (!valueType.isInstance(storedValue)) {
        throw new IllegalArgumentException("Cannot cast from " + storedValue.getClass() + " to " + valueType);
      }

      return valueType.cast(storedValue);
    } catch (Exception ex) {
      throw new KeyValueStoreException(ex);
    }
  }

  @Override
  public IKeyIterator keyIterator(IKeyPattern pattern) {
    try {
      if (pattern == null) {
        return new KeyIterator(hazelcastMap.keySet());
      }

      SqlPredicate keyPredicate = new SqlPredicate("__key like " + pattern.toString());
      Set<String> keys = hazelcastMap.keySet(keyPredicate);
      return new KeyIterator(keys);
    } catch (Exception ex) {
      throw new KeyValueStoreException(ex);
    }
  }

  @Override
  public IKeyPatternBuilder getKeyPatternBuilder() {
    return new HazelcastKeyPatternBuilder();
  }
 
  // implementation of other methods omitted

  private static class KeyIterator implements IKeyIterator {

    private final Iterator<String> hazelcastKeyIterator;

    public KeyIterator(Set<String> keys) {
      this.hazelcastKeyIterator = keys.iterator();
    }

    @Override
    public boolean hasNext() {
      return hazelcastKeyIterator.hasNext();
    }

    @Override
    public String next() {
      return hazelcastKeyIterator.next();
    }

    @Override
    public void remove() {
      hazelcastKeyIterator.remove();
    }

    @Override
    public void close() throws KeyValueStoreException {
      // this iterator doesn't need to be closed
    }
  }  

}

First thing to note from the set and get methods is that an IKeyValueStore works with Java objects. So while the key-value store may contain arbitrary bytes as keys and values, those that are read or written through this interface cannot be arbitrary. They must be serialized Java objects (where the representation depends on the serialization mechanism - JSON, XML, byte array when standard JDK Serialization is used or anything else). In this case, Hazelcast automatically takes care of the serialization for us. Depending on the key-value store of your choice, you may have to serialize and deserialize objects yourself. This point also highlights why we recommend to always separate the keys and values used by IKeyValueStore implementations from other keys and values that may exist in the key-value store. In this example, the keys and values are stored in a Hazelcast map which is injected in the constructor. This map is used exclusively by our key-value store implementation. Other implementations may use namespaces for this purpose.

Another key point is the keyIterator implementation. The IKeyValueStore must be able to list all keys or some keys which match a given pattern. The return value of this method is a Closeable iterator. The reasons for returning an iterator instead of a set of keys are:

Listing all keys may be an operation which impacts performance. For example, see the warning on the Redis KEYS command. When implementing this method consider the performance implications and consider incrementally loading the keys (if possible) instead of loading all keys at the same time.
In many scenarios, a set of operations needs to be performed on each key. So it is not necessary to have all keys up front, it is sufficient to have one key at a time.

The iterator is Closeable because some implementations may need to hold a connection to the key-value store open for the duration of the iteration. This connection should be closed when iteration is complete. In our example, the keySet() method may throw a QueryResultSizeExceededException, so this example is not completely safe to use on large data.

Implement IKeyPattern and IKeyPatternBuilder

The key-value store must, at the very least, support a simple glob-style query language for keys. For example Redis uses * as a wildcard for any string and ? as a wildcard for any character. Patterns containing these wildcards can then be used with the KEYS or SCAN commands to list keys.

Hazelcast uses % as a wildcard for any string and _ as a wildcard for any character (similar to SQL). Patterns using these wildcards can then be used with a "__key like" predicate to list keys.

public class HazelcastKeyPattern implements IKeyPattern {

  private final String pattern;

  public HazelcastKeyPattern(String pattern) {
    Assert.notNull(pattern, "Key pattern must not be null");

    this.pattern = pattern;
  }

  @Override
  public String toString() {
    return pattern;
  }

  @Override
  public int hashCode() {
    // omitted
  }

  @Override
  public boolean equals(Object obj) {
    // omitted
  }

}

public class HazelcastKeyPatternBuilder implements IKeyPatternBuilder {

  private StringBuilder builder = new StringBuilder();

  @Override
  public IKeyPatternBuilder literal(String value) {
    if (value != null) {
      // note: this replaces % with \%, and _ with \_
      builder.append(value.replaceAll("%", "\\%").replaceAll("_", "\\_"));
    }

    return this;
  }

  @Override
  public IKeyPatternBuilder anyString() {
    builder.append('%');

    return this;
  }

  @Override
  public IKeyPatternBuilder anyCharacter() {
    builder.append('_');

    return this;
  }

  @Override
  public IKeyPatternBuilder pattern(IKeyPattern pattern) {
    if (pattern != null) {
      builder.append(pattern.toString());
    }

    return this;
  }

  @Override
  public IKeyPattern build() {
    HazelcastKeyPattern pattern = new HazelcastKeyPattern(builder.toString());
    builder = new StringBuilder();

    return pattern;
  }
}

A few points are important related to these classes:

IKeyPattern implementations must implement equals() and hashCode(), as they may be used as keys in various internal maps
IKeyPatternBuilder implementations must escape wildcards in literals
IKeyPatternBuilder implementations must reset after their build method is called

Implement IKeyspaceMonitor

The IKeyspaceMonitor implementation is going to be used for concurrency control. It must be able to detect when a key is set, deleted or expired (evicted) from the key-value store.

public class HazelcastKeyspaceMonitor implements IKeyspaceMonitor {

  private final IMap<String, Serializable> hazelcastMap;
  private final Map<DispatcherKey, HazelcastEntryListener> dispatchers = new ConcurrentHashMap<>();

  public HazelcastKeyspaceMonitor(IMap<String, Serializable> hazelcastMap) {
    this.hazelcastMap = hazelcastMap;
  }

  @Override
  public void addListener(String key, IKeyspaceEventListener listener) {
    HazelcastEntryListener hazelcastEntryListener = new HazelcastEntryListener(listener);
    dispatchers.put(new DispatcherKey(key, listener), hazelcastEntryListener);
    Predicate predicate = new SqlPredicate(HazelcastKeyPatternBuilder.KEY_PATTERN + key);
    String registrationId = hazelcastMap.addEntryListener(hazelcastEntryListener, predicate, false);
    hazelcastEntryListener.setRegistrationId(registrationId);
  }

  @Override
  public void removeListener(String key, IKeyspaceEventListener listener) {
    DispatcherKey dispatcherKey = new DispatcherKey(key, listener);
    HazelcastEntryListener hazelcastEntryListener = dispatchers.get(dispatcherKey);
    if (hazelcastEntryListener != null) {
      hazelcastMap.removeEntryListener(hazelcastEntryListener.getRegistrationId());
      dispatchers.remove(dispatcherKey);
    }
  }

  // other method implementations omitted

  private static class DispatcherKey {

    private final String keyPattern;
    private final IKeyspaceEventListener listener;

    public DispatcherKey(String keyPattern, IKeyspaceEventListener listener) {
      this.keyPattern = keyPattern;
      this.listener = listener;
    }

    @Override
    public int hashCode() {
      // omitted
    }

    @Override
    public boolean equals(Object obj) {
      // omitted
    }
  }

  private static class HazelcastEntryListener
      implements EntryAddedListener<String, Serializable>, EntryUpdatedListener<String, Serializable>, EntryRemovedListener<String, Serializable>, EntryExpiredListener<String, Serializable> {

    private final IKeyspaceEventListener listener;
    private String registrationId;

    public HazelcastEntryListener(IKeyspaceEventListener listener) {
      this.listener = listener;
    }

    @Override
    public void entryAdded(EntryEvent<String, Serializable> entryEvent) {
      listener.onEvent(KeyspaceEvent.SET, entryEvent.getKey());
    }

    @Override
    public void entryExpired(EntryEvent<String, Serializable> entryEvent) {
      listener.onEvent(KeyspaceEvent.EXPIRED, entryEvent.getKey());
    }

    @Override
    public void entryRemoved(EntryEvent<String, Serializable> entryEvent) {
      listener.onEvent(KeyspaceEvent.DELETE, entryEvent.getKey());
    }

    @Override
    public void entryUpdated(EntryEvent<String, Serializable> entryEvent) {
      listener.onEvent(KeyspaceEvent.SET, entryEvent.getKey());
    }

    public String getRegistrationId() {
      return registrationId;
    }

    public void setRegistrationId(String registrationId) {
      this.registrationId = registrationId;
    }
  }
}

The keyspace monitor must be able to register or unregister a listener for a given key or key pattern and to notify that listener when that specific key or a key matching that specific pattern is set, deleted or expired. Multiple listeners should be able to register for the same key or key pattern and one listener can be registered for multiple keys or key patterns. Unregistering a listener from a key or key pattern should not affect the registration of other listeners for the same key or key pattern.

Expose your implementations

You must expose your IKeyValueStore and IKeyspaceMonitor implementations to the Blueriq Runtime as Spring beans. Additionally, you should define a profile to activate your custom plugin and add any external configuration options that may be required. In this example we expose all beans in a configuration class guarded by a profile. If you choose to annotate your implementations with @Component, make sure you also annotate them with @Profile or be sure they are not component scanned unless a profile is active. The Blueriq Runtime assumes there is a single IKeyValueStore implementation available. Accidentally exposing multiple IKeyValueStore implementations will prevent the Runtime from starting.

@Configuration
@Profile(KeyValueHazelcastStoreConfig.PROFILE_NAME)
public class KeyValueHazelcastStoreConfig {

  public static final String PROFILE_NAME = "keyvalue-hazelcast-store";

  @Bean
  public IKeyValueStore hazelcastKeyValueStore() {
    return new KeyValueHazelcastStore(hazelcastMap());
  }

  @Bean
  public IKeyspaceMonitor hazelcastKeyspaceMonitor() {
    return new HazelcastKeyspaceMonitor(hazelcastMap());
  }

  @Bean
  public IMap<String, Serializable> hazelcastMap() {
    return hazelcastInstance().getMap("blueriq");
  }

  @Bean
  public HazelcastInstance hazelcastInstance() {
    ClientConfig config = new ClientConfig();
    // add any external @ConfigurationProperties that may be needed
    return HazelcastClient.newHazelcastClient(config);
  }

}

Note that the IKeyValueStore and IKeyspaceMonitor implementations should use the same map. Additionally, this map should not be used for other purposes to prevent non-conforming data (eg. data which does not represent a serialized Java object) from being written to the map.

Update the configuration

Finally, you must update your configuration in order to use Hazelcast as a session store.

In boostrap.properties, remove the profile of the default key-value store component and add your own profile:

# old config
#spring.profiles.active=native,keyvalue-redis-store
 
# new config
spring.profiles.active=native,keyvalue-hazelcast-store

In application.properties configure the Hazelcast connection and configure Spring Session to use Hazelcast

spring.hazelcast.config=/path/to/hazelcast/config
spring.session.store-type=hazelcast
 
# this can remain unchanged
blueriq.session.session-manager=external

You do not need to change the blueriq.session.session-manager property. The external session manager implementation will be able to pick up your IKeyValueStore implementation and use it to manage sessions.

Previous: 2. About the load balancer

Next: 4. Cluster configuration

Space shortcuts

Page tree

Introduction

Redis Key-Value Store Component

Prerequisites

Enable/Disable

Properties

Using the key-value store in custom plug-ins

Using other key-value store implementations

Create project and add dependencies

Implement IKeyValueStore

Implement IKeyPattern and IKeyPatternBuilder

Implement IKeyspaceMonitor

Expose your implementations

Update the configuration

Space shortcuts

Page tree

3. Key-value store API and default component [editor]

Introduction

Redis Key-Value Store Component

Prerequisites

Enable/Disable

Properties

Using the key-value store in custom plug-ins

Using other key-value store implementations

Create project and add dependencies

Implement IKeyValueStore

Implement IKeyPattern and IKeyPatternBuilder

Implement IKeyspaceMonitor

Expose your implementations

Update the configuration