Model preloading

When a user makes a first request to a Blueriq application, the model that defines the application is loaded into memory. This makes the first request slower than the subsequent requests.

In some situations, it may be desirable to load some models into memory during the Runtime startup, especially when the model is large and takes a long time to load. An example is when you have a runtime cluster and use auto scaling you don't want the first request to a fresh new server to take a lot of time, because the model is not yet loaded.

You can control which models get preloaded during Runtime startup with properties described in Model Preloading Properties.

Considerations

Preloading models will remove the load time from the first request, but the load time will be added to the Runtime startup, so runtime startup will take longer.

Also, if you load all versions of all Blueriq applications, the Runtime might end up loading models in memory that are not used at all, so less memory is available for sessions.

Our advice is to preload models when:

• The model will definitely be used
• The model is big, so the user would have to wait for an unreasonable timespan in the first request

Preloading after accepting HTTP requests

By default, configured models are loaded before the Runtime accepts HTTP requests. This means that the load times of all models will be added to the Runtime startup time, but you will have the guarantee that all designated models are loaded once the runtime is started.

If you set the blueriq.modelpreload.before-requests property to false, models will be loaded as soon as the Runtime startup is complete and when it accepts HTTP requests. If there is a request to an application that has not been preloaded yet, preloading will still take place during the request.