Restoring logic to the heart of IS via ontologies and logic-based business rules
Abstract. The failure rate for the creation of critical and agile Information Systems (IS) is striking. Public sector examples include NHS in the UK and Louvois in France. However, the situation in the private sector is no different. Various approaches have been proposed by software professionals with limited success. At the same time, the use of logic — the ‘mathematics of software’ — has been abandoned in favour of ‘coding’ alone, nor has its application been put to good use for commercial developments. Furthermore, the achievements of artificial intelligence, albeit at first sight impressive, have further distanced software engineering from a fundamental approach based on logic. The ‘software crisis’ persists, whilst the increasing digital transformation of society raises significant quality, transparency and cost challenges. This article introduces the transformational contribution of formal logic — ontologies and business rules in logic — to the development of real-world IS in use.
This ‘ontology-centric’ approach to software development has been applied successfully to many business-critical projects. Contact us for more information.
Keywords: Logic, rules, ontologies, specification, Business-IT gap, software engineering, digital transformation.
The Software Challenges
Although hardware, networks and cloud computing continue to perform better at lower cost, software development remains a constant source of problems: the failures of the National Program for IT in the UK [NHS 2019], the Louvois and ONP projects in France [LMI 2014] and the Expeditionary Combat Support System [ECSS 2012] are notable examples. The private sector has also experienced major failures, even for projects carried out by leading software and service providers e.g. DHL [DHL 2015).
This apparent contradiction makes the real reasons for issues faced during many IT projects all the more obscure to company directors or the Minister responsible for them, especially as some of them or their children try to develop small-scale applications with varying degrees of success. Also, everyone uses multiple applications that work reasonably well and generative artificial intelligence seemingly works miracles.
Studies into success rates of IT projects [Standish 2015] are no clearer: they avoid analysing the root cause(s) of the ‘software crisis’ [NATO 1968] and instead make entry-level recommendations: get management on board, involve users deeply, clearly articulate requirements, plan the implementation in the best possible way, etc. These are undoubtedly necessary conditions for success, but are by no means sufficient, if not unrealistic.
A complete and precise formulation of requirements is possible to put a satellite into orbit: physics and mathematics can be used to formulate the problem precisely. This is not the case for a business application: obtaining a correct and consistent specification of the business problem is the main software development obstacle. As rightly observed by Frederick Brooks, Jr. “Much of the essence of building a program is in fact the debugging of the specification.” [Brooks, 1987]. This is the result of the ‘Business-IT gap’, where Business experts and IT specialists spend significant time, energy and money trying to understand each other.
Criticizing users for an incomplete and vague formulation of requirements ignores the obvious: it is up to IT specialists to provide the conceptual and technical tools enabling the business and IT to express, validate, refine and share a precise understanding of the problem, without which IT development is destined to fail.
Similarly, becoming irritated when the business changes the definition of the problem ignores the ever-changing nature of the business environment, especially for long projects making changes more likely.
Why Building Software is Difficult
The vast combinatorial complexity of software is the first difficulty to overcome. A large number of precise and complete technical details have to be added to the business problem description in order to build a program executable by a computer. Using today’s technologies, these programs can have hundreds of thousands or often millions of lines of ‘code’. The scale of such systems makes them literally impossible to test effectively. A highly combinatorial and therefore unstable system lies at the heart of the resulting ‘machine’.
The second difficulty is that an IS, like most strategic software, is not a variant of a system whose design and build has been replicated thousands of times over the course of history: it is a unique instance which may, at best, make use of a few technical design patterns albeit unrelated to the business.
Conventional (non-IT) systems, e.g. bridges, are very different. They are often stable over large value ranges. Testing can be limited to reference cases from which it is possible to know the overall system behaviour given the continuous nature of the physical laws that describe them. In addition, repeat builds of the same type of object over time enable the cumulative experience gained to be documented and leveraged. Finally, testing such systems validate a priori modelling based on physical theories, their mathematical expression and numerical models that make the calculation possible with the necessary precision.
The same cannot be said for large-scale enterprise software development: each is unique with no a priori ‘mathematical’ modelling (except for safety-critical systems, with their high cost and long implementation timelines) and where exhaustive testing is impossible, as the number of states to be tested often exceeds the number of seconds that have elapsed since the Big Bang (1 followed by 18 zeros!)
This explains the high failure rate of major IT projects, design issues and the high number of bugs that is characteristic of most software.
Complexity and Change
Business-critical software must cope with three types of complexity:
The complexity of the ‘what’ i.e. the problem to be automated. This is an essential (see [Brooks F., 1987]) complexity, linked to the essence of the problem and the project. It is irreducible since it results from the needs of the organization and the constraints it faces.
The complexity of the ‘how’ i.e. the complexity of the programming arising from the software technologies used. This complexity is accidental since it is specific to the technologies chosen within the constraints and limits of the software market offerings.
The complexity of integration. A software project never exists in a vacuum: it is integrated with existing software and technology. Here too one must differentiate between the ‘what’ (what are the semantics of what needs to be integrated) and the ‘how’ (what protocols and architecture will be used).
In practice, the project owner provides the project manager with an informal description (in the form of texts, tables and diagrams) of what the software to be built should automate. This description is interpreted by software developers and translated into data structures and programs. Business knowledge is coded using a programming language, designed to communicate with computers and not humans. IT specialists are exposed to ‘complexity squared’: the (essential) problem and the (accidental) technical solution. Since a precise and complete definition of the problem is unrealistic, it will evolve over time leading to changes in many lines of code with unpredictable effects that are difficult to test and correct, as business and technology are intertwined. The accumulation of changes increases the entropy of the code already written, making changes increasingly difficult. This explains why an error identified during acceptance testing is 50 times more costly than an error identified during specification (see next chart).
What Should Be Done?
‘Complexity squared’ (business and IT intertwined) must be replaced with two complexities that are managed separately. This requires a clear separation between the specification of the business ‘what’ from the realization of the IT ‘how’.
The specification of the ‘what’ must cover the totality of the business problem. No business aspect should be covered by the IT ‘how’.
This specification must be declarative: it must only express business knowledge, avoiding any reference to the ‘how’ and in a language that the business can understand.
This specification must be complete, unambiguous and testable before the first line of the ‘how’ code is written. This requires a ‘mathematical’ approach i.e. formal logic. During testing, the reasons for the results observed must be explainable. This ensures the problem is understood in depth and ‘zero defects’ at the conceptual level are achieved before the first line of code is written. Only an executable specification enables testing and explanation.
The specification must be an executable business model expressing 100% of the business logic. It must drive the application such that IT specialists can add what is not in the model (the technical aspects) without concern for the quality and complexity of the business specification.
Finally, as with modern industrial processes, the production chain for an application must be automated: the executable business model must be used ‘as-is’ directly by IT specialists with the required performance and the ability to scale up.
The above requirements can be achieved using W3C Semantic Web technologies, based on formal logic, combining ontologies (the conceptual model of the domain and problem with the associated vocabulary) and business rules (expressed in logic). The transition from the logical world of the business definition to the program-centric world of software developers must be automated.
We thus move from a craftsman approach (building software by hand in the same way the Romans built bridges) to a scientific approach (‘calculating’ software in the same way a bridge is modeled before building it via simulations such that it can be tested).
While this approach is still largely ignored in daily practice, it is not confined to the research domain: it is being successfully applied for business-critical applications. It is a true innovation that is providing a solution to the ‘software crisis’ challenge.
Real-World Uses Cases
This ‘ontology-centric’ approach to software development has been applied successfully to many business-critical projects. The last one is the development of the product and quote capabilities for a public national railways operator, deployed in the cloud, providing:
A high level of business agility.
Technical flexibility (architecture, infrastructure, etc.).
High performance in terms of response time and volume.
Near-zero-defect operation to minimize corrective maintenance.
The shortest possible development time.
All these objectives were achieved in six months by a small team of only 3.5 full-time equivalent.
A more detailed presentation of the project is available.
Other use cases include, among others, complex sales IS in the insurance sector (Langevine 2014).
Contact us for more information.
References
BROOKS, F. (1987). No Silver Bullet: Essence and Accidents of Software Engineering. In Computer, Vol. 20, №4 (April 1987) pp. 10–19 http://worrydream.com/refs/Brooks-NoSilverBullet.pdf
DHL (2015). DHL finally abandons ill-fated NFE IT project and is forced to write off €345m https://theloadstar.com/dp-dhl-finally-abandons-ill-fated-nfe-it-project-and-is-forced-to-write-off-e345m/
ECSS (2012). Air Force scraps massive ERP project after racking up $1B in costs https://www.computerworld.com/article/2493041/air-force-scraps-massive-erp-project-after-racking-up--1bin-costs.html
LANGEVINE, L. (2014), BONE, P. The Logic of Insurance: an Ontology-Centric Pricing Application. Proceedings of ISWC 2014, October 2014. https://ceur-ws.org/Vol-1383/paper2.pdf
LMI (2014). Après Louvois, le gouvernement enterre le projet de logiciel de paye ONP. https://www.lemondeinformatique.fr/actualites/lire-apres-louvois-le-gouvernement-enterre-le-projet-delogiciel-de-paye-onp-56816.html
NATO (1968). Software Engineering. Peter Naur and Brian Randell eds. http://homepages.cs.ncl.ac.uk/brian.randell/NATO/nato1968.PDF
NHS (2019). Case Study 1: the £10 Billion IT Disaster at the NHS. https://www.henricodolfing.com/2019/01/case-study-10-billion-it-disaster.html
Standish Group (2015). Chaos Report — Q&A with Jennifer Lynch. In InfoQ, https://www.infoq.com/articles/standish-chaos-2015