In the current pipeline construction process all of the data is collected in the form of records that are compiled and attached to in a document control system. Often times there are delays in information being processed leading to additional costs which gets increasingly expensive as time passes by.
Our vision to solve this issue was to collect the data in an actionable format that would stream the data to a central location. As the pipeline data was uploaded project updates for all aspects of the project would be achieved in real time. There are numerous challenges faced in this type of project.
We utilized a relational database to build the software. When dealing with a high volume of transactions, while also handling the flexible data structure that the project required a graph database such as Cosmo would have been the better option.
Once again, the data model that was selected ended up making it difficult to get data out of the system. There was a significant amount of overhead in the relational database to support the flexibility we needed. This resulted in complex queries and pivots to get data out of the system. This issue would have been solved using a different database.
The goal behind assets was to provide 360 degree visibility into an asset. This included all financial aspects of the asset, maintenance activities and any live data collected. Our first attempt at collecting live information was to connect the sensors in the field directly to the application. This resulted in scalability issues. To solve this we utilized the Azure IoT Hub to first stream the data and then commit to the database. That significantly improved our ability to handle live data streams.
Our proof of concept for the project was highly successful. We were able to successfully develop a minimally viable product to prove out our theories.