My Profile Photo

Cooking Analytics

My name is Miguel Gaspar and this blog will be focused on Analytics, providing architectural and technical solutions. I will provide "recipes", that can be applied to Big Data, IoT and Social Innovation envolving edge technologies. Hope you like it.

Update to Realtime dashboard and PDI transformation sample using MQTT

More than 1 year ago I have released a series of blog posts about a real-time dashboard using CTools and MQTT protocol. This covers how we could build a real-time dashboard for IoT showing real-time sensor data in a CTools dashboard.

The posts also show how we can perform data streaming in PDI, by using non-ending transformations (and don’t try to do it with jobs, at least not for now).

Here are the links:

The samples I’ve created were using some custom code on PDI, but in the meanwhile, Pentaho Labs released MQTT steps on marketplace.

I have updated my samples to use the MQTT steps in Pentaho Marketplace, and also updated the libraries used of the frontend. The samples are simpler to execute since I am now using a public MQTT broker.

You can download them from the link

Instruction to get them working:

  • Download the files;
  • Run iot-device-emulator.ktr and streaming-using-mqtt.ktr kettle transformations available inside the backend folder;
  • Upload the files inside frontend folder to your Pentaho server;
  • Run the CTools dashboard.

At this point, you should be able to see the charts and map being updated automatically without any pull request. All data is pushed directly from Pentaho Data Integration and the broker using the MQTT protocol.

Follow me at Twitter

Twitter LinkedIn Facebook Google+