Evolving Water Operation’s Edge With Ignition And MQTT

00:00
Jaco
My name is Jaco. I look after the team here at element eight. We are the Sub-Saharan African distributor for ignition. And we are very excited to be able to share some local success stories with you from South Africa and from Africa. And our first success story, participation as part of the ICC. So if you’re not sure what the ICC is, you’re probably in the wrong place. You already joined the session. The ICC is Ignition’s community conference, annual conference. Prince. It is, of course, now in 2021. It is virtual again, following the back of the first virtual session last year. And, yeah, the good news is that at least we’re able to bring you some success stories and some journeys from people locally in South Africa that we typically probably wouldn’t have been able to do if weren’t with you in North America.

00:46
Jaco
But are we looking forward to the session? So, as I said, my name is Jaco. I am with me, I have Brian Cooper. Brian Cooper is a director at INTEG Systems Integration. Gold certified Si. Brian, good morning. Welcome.

00:59
Brian
Hi. Good morning, Jaco. Happy to be here.

01:02
Jaco
I had to add the gold certification on there. I think you were able to achieve that gold certification fairly quickly, so well done on that. So we’re chatting with Brian this morning. As I said, INTEG is a gold certified system integrator. We’re chatting this morning. The title is evolving water operations, edge with ignition and MQTT. So if you look at the title, there’s a couple of giveaways there. We’re definitely going to talk about the water industry or one of Brian and INTEG customers in the water industry. We are talking about edge. This is an edge solution, and it was done using ignition and, of course, the MQTT protocol. So it’s a really nice solution put together. But before we look into that, Brian, maybe you can tell us a little bit more about INTEG and about Oudtshoorn, who’s your customer in this case.

01:50
Brian
Yes. Thank you, Jaco. So INTEG is a system integrated company. We specialise in PLC and HMI applications and more complex SCADA and OEE reporting type applications as well. For the last couple of years, we’ve also been involved in the IIoT industry specifically for water monitoring, water infrastructure monitoring and control. And the ignition product really has opened up a lot of doors for us with the way that it’s structured and the way that it works. So we’ve sent a few guys on training to get us onto par with the platform, and, yes, we’re very happy to implement it. Oudtshoorn is a municipality in the Klankarua region of the Western Cape. We’ve been involved with them the last four years.

02:50
Brian
In the beginning, they contacted us just to investigate the possibility of remote control and remote monitoring some of the booster pump stations and reservoirs and so on. And over the years, the system has grown to a very large system where we monitor all of the infrastructure, the dams, the flows between the dams and the town reservoirs, the town reservoir levels, bore holes, they’ve got a whole network of booster pump stations, there’s a water scheme, there’s weather stations, these water purification plants, and so on. So we’ve been involved in getting all of this information onto a platform for them where they can remotely monitor it, and we’ve had some great success in that as well. Oudtsburg is the municipality.

                                              
03:45
Brian
The area where they are situated is very dry at the moment, and they’ve got very limited water resources, so it needs to be managed in a very careful way. And I think that’s where we added a lot of value for them over the past couple of years.

04:00
Jaco
Cool. That sounds good. So, 100%. I think water scarcity is something that’s definitely a reality in Africa, and especially this part of southern Africa. So the ability to monitor usage levels at a fairly good speed and a higher uptake is very important to be able to make decisions in terms of what to open, what to close, what to fault away. So if I understand this correctly, this was primarily a monitoring solution of the various operations that they have in place to be able to get a heads up in the event of, for example, a low status or a low level or a heads up in terms of any flow that is potentially under or over limit. So it’s a monitoring solution primarily?

04:46
Brian
Yes. Initially that started out as a monitoring solution only, but as time went by, they had the requirements to remotely start and stop boreholes as well, because of the remote installations or the remote locations of these boreholes. So the system grew and expanded into a system that they can remotely control as well.

05:11
Jaco
And how would they typically do that prior? Would they physically drive there in a vehicle?

05:16
Brian
Yes, they would physically drive there in a vehicle. Roads are not very good, so you need a bucky or a truck with a lot of clearance. And also it takes you about 2 hours to just get there to the furthest borough to switch it on, drive back out again, and then in the afternoons, they had to drive back in to switch them off again. So it took an enormous amount of time and obviously wear and tear on the vehicles and labour rates and all of that to control these borders.

05:48
Jaco
Okay. All right, that’s good context. Thank you. So the solution that you ended with or that you’re still busy implementing, I think probably this journey is evolving and you’re building out this application as you go. But you settled with ignition and MQTT. So just around ignition and MQTT, for those of you that are watching that’s not familiar with MQTT, is obviously an IoT protocol, messaging protocol that we’ve spoken about often and quite a lot. So your ideas around MQTT and ignition, why did you initially think of those two protocols in this specific case? Can you highlight some of the advantages?

06:29
Brian
Yeah. So on ignition, the reason why went with ignition was because of the fact that you don’t have additional cost for view nodes and because it is a platform that’s built to be installed as a remote system or a cloud based system where you can access the system from anywhere and then give access to anybody on that system. Traditional skaters are not that easy to do that with. So ignition lends itself to be implemented in that way. Also, because ignition is open, it enables us to do a lot of scripting and stuff in the background to manipulate some of the data and to make it really user friendly. For our client, we used MQTT because it’s a lightweight protocol. It’s built for remote type applications as well.

07:31
Brian
So specifically the spark plug B version where we make use of the store and forward capabilities as well. So we wanted something that’s lightweight because we have a lot of applications where we need to install gateway with a SIM card and we need to take the auto cost into consideration. So the lightweight and report by exception made that a good protocol to go with. The second one is we often have load shedding or power failures for long periods of time, even to the extent where cell phone thousands and that type of infrastructure goes down. So we need to be able to lock the data locally and store them forward. When that communication comes back up again, we’ve got solar sites and so on that keeps on pumping. So we need to get that information back as soon as the power comes back on again.

08:29
Brian
So that’s another reason why we use that. And another reason is also now we’ve got a unified namespace where we can build the structure of a complete plant, of a complete municipality’s infrastructure, and we can give access to other systems as well that needs access to this information. Awesome. So yeah, that’s why we use the ignition and QTT.

08:58
Jaco
So you mentioned report by exception. Could you maybe explain briefly, if you don’t mind, sort of the difference between reporting by exception and kind of a polling protocol that probably 80% of us are used to.

09:12
Brian
Yeah, so the polling protocol, you normally have a server that polls all your devices, your clients in the field. If it doesn’t get a feedback from that device, it’ll poll it again and poll it again, and it puts additional overhead onto your network. Report by exception. Also, it polls for all the information all the time. So if you get all your clients to give feedback, you can easily overload your network. And with remote systems where we use 400 mhz radios, your bandwidth is not that big. So you need to manage your network load and your network data load to keep it there as low as possible. Report by exception is basically your edge device or your borrow in the field will only send the information or a level of the borrow once that level has changed, or it will only send the start.

10:15
Brian
The pump is running once that pump has started. So you only send the information that’s necessary to be sent, and you only send it when it changes. So it reduces your network loading considerably, and it doesn’t send everything each time. So even if there’s only a small change, it’ll only send that small change. It won’t send all the whole data packet. Okay, so it makes a lot of sense, specifically when you want to keep your cost low, your data cost low, and your data traffic low and so on.

10:47
Jaco
For sure that reporting only when there’s a change, combined with the lightweight. So when we say lightweight, I think we’re talking about MQTT Sparklab D. I think were literally talking about a two byte header, extremely lightweight report by exception, as opposed to the polling that generally just kind of sweeps for any changes. And the other one that you mentioned, that’s quite important. We’ll be talking about the overall MQTT architecture is the unified namespace. We’re probably not going to have enough time, at least in this session, to talk about the unified namespace. But the unified namespace is a concept, and I think MQTT as a protocol, together with the broker centric approach, provides a lot of the foundation that you need for the unified namespace.

11:34
Jaco
And to your point, that’s something that you have in place now that can help you build out to other parts of the network.

11:41
Brian
Yeah, you’re correct in saying that it’s a concept. There’s a lot of different ways to implement it and to interpret it, but it’s an interesting concept and it definitely makes a lot of sense.

11:54
Jaco
Cool. So we’ve got on the topology on the screen, you have the server, obviously it’s MQTT. Is that a MOXA gateway that you’ve used?

12:04
Brian
Yes. So this is a very simple interpretation of the system that we’ve installed there. So on the edge we’ve got the Al Pro radios, which is 100 meg radio communication to a base station or a repeater. And then from there we’ve got a MOXA gateway, which we’ve installed ignition edge on. And then we’ve got the MQTT protocol from there to our cloud hosted server, and then we’ve got clients that connect to the server to visualise.

12:41
Jaco
All right, so that’s a Moxa gateway. That’s a stock standard MOXA UC kind of gateway that’s available. Did you load it with any extra memory or anything like that?

12:53
Brian
No, we didn’t load with extra memory. So we’ve got the UC 8200 I think we sold there, and we got the bigger version. I can’t remember the exact part number, but the bigger version that’s got enough memory to load the ignition edge on. We had to do some firmware modifications because the version that we get here is not exactly the same as the US version where they sell it with the ignition edge software on. So we had to do some firmware modifications to get the licensing and stuff to work perfectly fine. But apart from that, it’s basically stock standard with ignition edge on it and works perfectly perfect.

13:38
Jaco
Then, just looking at the next slide, I think this probably shows a little bit better in terms of. So you’re currently using LTE or GSM for the communication?

13:50
Brian
Yes, so we use GSM. So I think those two lines just shows you that it’s a decentralised system. That’s what NQTT gives you, and that makes it easy to implement in a wide area or a big system. You don’t need to have a WAN network set up with a lot of routing and network type configuration. MQTT allows you to put the system together that works perfectly.

14:29
Jaco
Fantastic. Apart from it being quite lightweight. What I like about MQTT is the sort of the scalability. Wherever you have a windows or a Linux or an arm, in this case an arm kind of a gateway, wherever you have one of those, you could literally put down an edge IoT license and immediately that can form part of your architecture and help get data across. I like the lightweight aspect of it as well.

14:56
Brian
Yeah. So on the outswing application, we actually had a very big 400 meg network that started to get a bit overloaded because of all the edge devices we had on there. So we are slowly installing edge devices on the traditional repeater sites so that the devices can talk directly to the edge device and then to the cloud broker instead of going through a bunch of repeaters to one central point. This allows us to build some redundancy into the system so we’re not reliant on that repeater backbone anymore, or only reliant on that repeated backbone anymore. And we’ve got better communication to different sites.

15:41
Jaco
Cool. And then I think on this screen, this is the one that I really like. We always talk about mobility and about the mobile view and extending the application literally to somebody’s cell phone in their hand. I really like this overview screen as well as what the mobile looks like. It’s not designed that way, I’m guessing, but there’s almost a little bit of situational awareness in there. Unless it was by design. But maybe you can talk us through these slides, please.

16:10
Brian
Yes. Okay, so the slide on the right hand side is basically the desktop view. So they opted for a maps view where each one of those pins indicates a site, a borrow or a reservoir or a booster pump station, something like that. And those pins will turn red if there’s an alarm or a fault condition on any one of those sites. On the left hand side, you have a list of all the sites with a simple status that shows you if the pumps are healthy, if they are running, what the level is, what the flow is, and so on. So this is a very simple way for them to easily see what the status is of their site. So we tried to implement the situational awareness here as well. And then on the mobile side, it’s a lot different.

17:11
Brian
Developing a SCADA for desktop and mobile. On mobile, you can’t have the PNIDs or the big overview layouts of your system because it’s just too small to view and to actually see what’s going on the mobile. So we decided to have this list view that gives you the flow and the levels and the pump statuses with some context, with that spark line, just to give you some information. So if you have a look at it with one view, you can see exactly what’s going on with that specific site, if it’s healthy, if the reservoir level is dropping, and if they need to do something about it, and so on. If you click one of those sites, it opens up a little pin showing the two pumps, the suction delivery pressures and the flows and the reservoir levels and so on.

18:01
Brian
So that’s quite nice for them to then click on a site to see where the fault is, why there’s a fault, and what they need to do to fix it.

18:09
Jaco
It’s amazing. I love the simplicity and the design. I mean, it’s clean, it is neat, but it contains just enough information to not confuse and enough information that you would need. As far as the overview of the trends and the status is concerned. Very nicely done.

18:26
Brian
And it’s very interactive and very simple to use. So it’s something that we developed in such a way, or tried to develop in such a way that even without training, we give it to one of the clients, they open it up and they can immediately navigate and use it. And it’s easy to use, which is also an important aspect of the stuff development. So on the mobile view, we’ve also got a little trend page where they can quickly have a look at some more detail of a specific analogue or digital that we are utilising. But then on the desktop view, they can obviously use the ad on trends tool and do some more in depth analysis of data and so on.

19:07
Jaco
It’s quite a lot of information you were able to fit into the screen. If I look at the mobile screen, you’ve even got a little spark line in there. That’s a lot of information to represent in a very small size real estate on a mobile phone.

19:22
Brian
Yeah, it is, but it gives them a total overview of the whole system. And then also it gives them that little bit of context, which is always important. An instant level or a point level of a flow or a level or whatever. It’s difficult to see what it’s doing. Is it going up, is it going down? Was the flow constant over the last couple of hours? Or is it jumping around so they can easily see if there’s something that they need to react to?

19:55
Jaco
That’s the value of that line. I think the line to your point is you may not necessarily understand if it’s trending upwards or downwards without that line. So that’s exactly something simple, but quite effective to put in there. And then I noticed the radio stats, the device specific monitoring, how is that exposed to ignition or where is that made available.

20:19
Brian
Okay, so that is the ALPR radio information. So each edge device has radio information on it. So we need the channel utilisation over the 400 meg line, and we need the signal strength and then battery voltages and uptime and so on. So that gets pushed from the edge device, from the Alpha radio, which is on the edge to the gateway, and then we just pull it up into ignition, and then we historise that information. So that information is also very important to do diagnostics on the radio network. If the channel utilisation goes up for some reason, we can trend it. We can also see that on our trends and we can react to that. If the signal strength goes down or decreases, we can act on that.

21:09
Brian
This also generates an alarm on that specific station, so we can see whether that station is online or offline and so on. So that’s more for diagnostic purposes, but it’s really valuable.

21:23
Jaco
Very nice overview. Just one last question. On the GIS, on the map embedded there, what are you using to do that? Is that a live GIS or is that a map view?

21:41
Brian
That’s live GIS.

21:42
Jaco
Okay. All right. I know there’s a couple of ways to do that. I’m sure there would be a hint in the screen or one of your clever guys would be able to tell us. But I like that live.

21:55
Brian
You caught me out on that one. I did that.

22:00
Jaco
Cool. So we’ve spoken about MQTT and why it was ideal in your case. I think just to recap, apart from the lightweight, and the sort of almost want to call them the bandwidth friendly capabilities of MQTT, I think the scalability to your point is as you are progressing and working with this system and this network, you basically just add wherever you need devices and MQTT capability, you add the components to get it. That really does help the scaling capability. The network bandwidth consumption being very light, I think is definitely a big one for most people looking at MQTT, that obviously to the next point makes it quite well suited for remote sensing and control. The one thing that we’ve spoken about, the reporting by exception, the one thing that people often wonder about in these cases is the security.

22:58
Jaco
I know that MQTT does come with a standard kind of TLS. All the necessary authentication and everything else that you would imagine forms part of an IoT application in 2021. Was security a consideration for you? Did you look at security specifically? And naturally, did it tick the box for you?

23:19
Brian
Yes, definitely, because it’s a remote application that has access to cell phone towers and so on. So the data travels through there. So we need to have a look at that. So we’ve got TLS implemented, even the Alpro radios have security on that radio network. So we’ve got TLS implemented from the edge straight through to the gateway and then up to the hosted scalar as well. So that was definitely a big consideration.

23:49
Jaco
Okay, perfect. I think what we’ll do is as part of the summary slide for this, I think we’ll maybe share a couple of links to the specific devices specifications around the hardware that you used. I think maybe that’ll be useful for people to understand exactly how you did that. So just as a project overview, it’s roughly about 2000 plus tags, 15 screens. Obviously that is led by the primary overview screens. And you’ve allowed for a couple of pop ups and kind of sub screens that supports that primary screen. You have the two fixed clients, and the two fixed clients is, I mean, it’s not a high number of clients, but I think the big benefit or the big view here was the mobility. And you’re serving eight mobile clients with this.

24:38
Brian
Yeah, I think the way that these systems are being managed, the guys are not sitting in front of a control room screen 8 hours of a day or 24/7 something like that. They are in the field and they need to have access to the system. And I think that’s where a lot of the benefit comes. So the technician can now be in the field, he can open up his cell phone, you can have a look at the whole system and see what he was. So the skater control room moved from the control room to his cell phone in the field, where he’s at most of the day. So we see that the mobile clients will be growing and not the fixed clients.

25:19
Jaco
Absolutely, you’re 100% correct. The remote field worker is a mobile one. It’s not somebody that specifically sits in front of a screen or a desk anymore. The mobility aspect was a big driver here. Cool. So just finally to wrap up in terms of, we understand that Oudtshoorn and the kind of feedback that we’ve seen, we understand that they’re very happy with the solution, the remote view and the capability, but they’re super excited, or they were super excited, still are, about the update times, could you maybe tell us what that means?

25:55
Brian
The previous system had long delays when they tried to start or stop something or reset something and so on, and that reduced significantly. Also the time it takes for a screen to open up and refresh and have the real time data there is now also significantly faster. So when they open up the app, the data is immediately there, it’s responsive, and it’s easy for them to use. They don’t have to sit and wait for stuff to update and refresh and so on. So that’s a big benefit for them.

26:35
Jaco
And naturally that person, they would typically get into, as we call it in South Africa, Bucky, which is for the rest of the world, literally an off road truck, that local person, they would typically get in that vehicle and drive to a pump to either go check what’s happening there, or turn on and off something. I mean, that’s a massive time saver for that person, as well as a cost saver.

26:58
Brian
Yeah. So they’ve reduced quite a significant amount of maintenance cost because of this. So they boosted pump stations. Now they typically go out there to clean the place up once a month, where traditionally they had to go out there to start and stop pumps. They had to go to drive to each reservoir to see, to check the levels, to make sure that they have water in them and so on, where now they’ve got visibility of the whole system, and they can just do it from a mobile phone.

27:31
Jaco
And then the last one that you have on the slide is the one that we probably didn’t, or probably one that you didn’t anticipate looking at the needs in terms of what you wanted to achieve. I think the ability to detect water theft and pipe bursts, that wasn’t necessarily something that was part of the list of requirements, but that is something that you have at the moment.

27:52
Brian
Exactly. And that’s something that they actually brought up a while. Well, afterwards, quite a while afterwards, they said that they detected a pipers, and they could show me exactly where that pipers was, because the system doesn’t show you exactly where it is, but because they know where the pipes run, they can look at the pressures and determine how far away from the pump station it is. And then also theft. They look at the pressures and the flows and the reservoir levels and the usage, and they can determine if it’s normal or not, and then determine if there’s the possibility of theft on those lines which they have detected. And they’ve actually caught a few guys, which is interesting.

28:35
Jaco
Amazing. That’s definitely a nice benefit as part of the system. Brian, thank you so much. I hope that wasn’t too fast. I think we’ll share maybe a separate list of just some of the devices. I can imagine a couple of people were dotting down and writing down things quite furiously. We’ll share the specifics around the devices that you used, but thank you so much for your time and for sharing the story about Oatesweren. We’ll be sure to include your contact details and if there’s anybody that wants to reach out with you about anything specific. And hopefully next time we can do this live in person with a much bigger audience. But I think it is extraordinary times that we’re living in. So thank you for jumping on Zoom and sharing with us. Virtually perfect.

29:20
Brian
Thank you very much for the time, and thank you for allowing us to be part of this. It’s exciting, and we really feel there’s a big place for ignition in our portfolio, and we’re very excited to be using this platform.

29:38
Jaco
Awesome. Brian, well done on the gold certification. Again, I think there’s only five gold certified in our region at the moment, so that’s a big achievement. Well done on that. Thanks, Brian.

29:48
Brian
Thank you very much. Keep well. Bye.