Business Intelligence (BI) – A brief history
The earliest known use of the term business intelligence is in Richard Millar Devens’ Cyclopædia of Commercial and Business Anecdotes (1865). Devens used the term to describe how the banker Sir Henry Furnese gained profit by receiving and acting upon information about his environment, prior to his competitors:
Throughout Holland, Flanders, France, and Germany, he maintained a complete and perfect train of business intelligence. The news of the many battles fought was thus received first by him, and the fall of Namur added to his profits, owing to his early receipt of the news.
— Devens, p. 210
The ability to collect and react accordingly based on the information retrieved, Devens says, is central to business intelligence.
This principal still stands very much true today but with one major difference. Organizations are under continual pressure to implement BI solutions faster than ever and be able to adapt existing solutions to meet the changing needs of a dynamic and competitive world to maintain their competitive edge. Traditionally, this is two areas where BI solutions have struggled, with long lead times from project initiation to delivering value common along with their inability to implement changes in a timely fashion.
Traditional ETL Tools
Most traditional BI solutions are designed to load data on some sort of periodic basis with the frequency of this determined by multiple factors, for example the availability of the source data. Data sourced from on premise databases, for example, can be queried on a regular basis if it doesn’t impact operational performance. Data supplied by external vendors on the other hand may only get delivered once a day as a csv file via sftp. In this case there’s little that can be done to improve latency.
Once loaded, information can be used to develop Key Performance Indicators (KPI’s) which are the critical (key) indicators of progress toward an intended result. KPIs provide a focus for strategic and operational improvement, create an analytical basis for decision making and help focus attention on what matters most. Interpretation and actions based on these KPI’s are made by analysts, for example comparing current performance to targets and thresholds or by looking for signals in the control chart, which indicate normal variation (noise) and exceptional variation (signal).
Reverse ETL – The final piece of the puzzle
Although data warehouses have been around for decades, it’s fair to say that most organisations have struggled to make the most of these incredible resources because getting information out, and back into other operational platforms, where users can utilise, it has been so hard. To solve this problem, a new component of the modern BI stack has started to emerge called Reverse ETL. This is effectively the opposite of ETL/ELT solutions where instead of moving data into the data warehouse, it’s moved out and into the operational tools that business teams rely on.
Enterprise Automation Platforms – Putting all the pieces together
Workato is a low-code enterprise automation platform that enables democratised process automation. It is a cloud-based platform that enables faster integration with legacy systems and interfaces. Automation is a key aspect of business success, and modern automation platforms are low code and easy to use. They can be implemented by the people that need it, on a technologically agnostic cloud platform with guardrails in place to keep your business and data safe, they are therefore more accessible to business users without the necessity for expensive IT buy in and support.
Workato, is a market leader enterprise automation platform providing pre-built connectors, business rules, maps and transformations that facilitate the extraction, transformation and loading of data as well as the ability to orchestrate integration flows so all that pain of trying to understand 10 different APIs suddenly disappears! Workato is a cloud based service and therefore consumers are agnostic to how these components have been implemented and only need to focus on their behaviour and how to utilise them to solve their specific business problems rather than the subtleties of how to implement them.
Often, APIs in this new distributed landscape allow us to subscribe to events and build workflow automations by “listening” to those apps for business events and once the conditions for the trigger get met, data can get loaded into the warehouse on a real-time basis.
And finally, to fully reap the benefits of a modern data warehouse, information and it’s supporting data needs to be made operationally actionable. This means that instead of just developing KPIs which support strategic decision makers. We can also leverage our data warehouse to drive insights at an operational level.
As an example, imagine you’re a service provider and you want to determine what the best campaign strategy is for loyal customers which is defined as hitting a certain level of consumption based on current market trends. You could easily setup a KPI that returns this at a customer level as soon as it’s recorded in the warehouse. Another recipe monitoring this KPI could then randomly subscribe the customer to one of a set of predetermined campaigns in Salesforce as soon as the KPI has been met. From these further analytics could be derived over time to determine the best conversion rates by campaign.
Conclusion
Automation platforms such as Workato are at the bleeding edge of this new paradigm, where specific recipes can be created, not only to load data to the warehouse but also to listen for events that potentially affect KPIs, and act based on their value. All without users having to remember to login to their BI reporting portal and interpreting the KPI dashboards for themselves.
Here at Cloudorizon, our expert team are passionate about helping companies maximise the value they can get from their existing systems as well as optimising business processes by introducing them to cut
ting edge integration and automation technologies that quickly add value.
