Posted By Jakub Koperwas, 04 July 2016
This is an initial post of an article series about Drools as a business rules engine. Instead of focusing on the complete syntax, API, features etc. in these blog series I would like to raise some other topics not being mentioned often, however crucial in the subject matter.
- addressing the real need to use this technology; pros and cons, possible scenarios and alternatives available on the market,
- how to design a proper fact (domain) model,
- what constraints should be put on the rule solution design,
- how to assure QA,
- caveats of the Drools engine mechanics,
- best practises on many levels (from development to an organization level),
- some basic info about the syntax and API will be covered as a necessary introduction to some more advanced topics.
The promise of a rule engine
The rule engines together with business process engines are all about the change and business agility.
Both processes and rules make up so called business logic, and need to be changed as soon as possible when the business conditions change, to assure full business agility. The rules are more about decisions, policies etc. whilst processes are the about workflow. Very often they are used together, when the flexibility in both aspects is required, however they need to be managed separately.
The main promise of these technology groups is that one will extract the core business logic from your current application or applications and put them in the separate repository. When the business needs to change — your rules (or processes) can be changed immediately by the business analyst (instead of a programmer) and immediately propagated to the runtime applications, (without the need to rely on IT operations department).
To be concrete — imagine an insurance company that basing on the potential client data (age, health condition etc.) and actual company products (like personal ins., family ins., etc.) tries to determine the final price for the client. When the business conditions change (i.e. company wants to introduce new a product like “family and pets” or a new health condition is from now on considered more risky, or some legal regulations change) the risk analyst can immediately change rules on a runtime system.
But is this promise really kept? Sometimes it is, but it’s not that simple and it has a price of its own. Unfortunately some customers I have worked with took this promise literally at first and for them Drools usage caused more problems that it solved.
Let’s demystify the promises and add some extra considerations.
Rapid deployment bypassing IT operations
Here comes the main problem. When bypassing IT operations, you also bypass existing QA processes, whilst the business rules represent the core business logic and therefore should be tested even more carefully then some application logic of less importance. This aspect needs to be developed carefully in all rule-related projects. Moreover the Drools nature does not ease the testing process but rather opposite. I will discuss QA in detail in some articles further on.
Rules edition by business staff
In the ideal world the business can change the rules itself without relying on a programmer, which also reduces the risk of misunderstanding of the specific requirements. (Once I have been involved in developing software for fund investors based on rule language, and could hardly properly recognize all the terms like Net Asset Value and other business related terms that were evaluated in rules engine).
However the reality is that the rule engine mechanics is not very intuitive for non-technical staff. Moreover the Drools (technical) language is difficult, so very often the programmers are still the only ones to change the rules. User friendly rule editing tool and — what is more important — additional layers of abstraction can be introduced like Domain Specific Language (DSL), or a spreadsheet to make the rules editing business friendly. However it only reduces rule language complexity. To hide the rule engine mechanics complexity, the domain model behind additional layer of abstraction needs to be developed with care. Such steps will result however in (intended) reduced flexibility, so again, the focus should be put not to diminish the advantages of a rule engine.
When the deployment, QA and abstraction layers are considered the BRMS (Business Rules Management System) is the software to address these issues (among the others). A good design is however still to be developed and some technical rules are to be implemented by a programmer. Besides that BRMS is another very complicated 3rd party software to be customized, maintained and very often not fitting the specific organizational needs.
The last but not least are the architectural considerations on further domain model design. Drools is a Java-related business rule engine, therefore it works in a business logic tier. Rules operate on the facts (i.e. data) that may come from the user interface, database or any other external data sources, or any combinations of these. And here come the three possible scenarios:
- one has many interdependent rules that work on a few facts, obtained e.g. from the user (like insurance fee calculation basing on client data like health condition, where the same health condition may be used in different rules),
- one has a few interdependent rules that operate on many facts obtained from external datasource, the facts can be processed sequentially however aggregate functions like: min., max., etc. may occur and rules may share common conditions (like segmentation of all customers into 4 groups, basing on their transaction history, each customer can be evaluated independently),
- one has many independent rules and many facts that cannot be evaluated independently (for example anti-money laundering system that needs to evaluate many rules with many accounts at once, or using aggregating functions like: min., max., etc.)
The first scenario is the easiest one. It does not require any special architectural considerations, as the fact can be simply loaded directly from user interface. Rule engine matching algorithms can easily match many rules with small amount of facts. It works especially well when the rules are sometimes interdependent, then algorithms involved in matching rules can perform better in comparison to a series of if statements in Java code.
The second scenario is a bit more difficult. All data from database needs to be loaded into business logic tier that will typically run on some application server. Fortunately, facts can be processed sequentially. The software developer needs to implement sequential loading from the database, and all aggregate functions calculation that are all data-wide need to be eliminated from rules and implemented to the application side and populated into facts variables. Nevertheless this scenario introduces serious performance overhead if large amount of data needs to be calculated at once.
The third scenario is the toughest one. There is no need to pre-calculate any aggregates on the application logic side as the cross-data validations are more complex and are subject to a frequent change. In this scenario only loading big portions of data (if not all) into an application server memory seems to be possible. Moreover, rule independence causes that the rule-matching algorithms does not introduce any reasonable optimizations. This scenario introduces disastrous performance. One of my clients reported a problem with his code, which needed 24 hours to perform the required calculation.
So what’s the solution for this case? Simply, do not use Drools. Such calculations need to be done in the storage tier. This certainly does not make the rule approach unsuitable in general, however one needs to find a rule-engine that works in the storage tier or implement its own (which is often the case) using e.g. PL/SQL procedures (which has the advantage that is more commonly known language).
When to use Drools and what are the alternatives
Do you suffer from frequent business logic changes and despite the fact that you develop iteratively, because of IT operations the releases are not as frequent as you wish, and you seriously consider business rules language implementation?
Consider adopting DevOps first as it addresses that problem in a holistic way (not only business logic).
Do you need to manage business logic deployments in a very specific way — for example you need to keep old versions of business logic in production (because for example some of your customers works on the old version of tariffs etc.)? You should probably follow the business rules route, however:
- consider architectural conditions in which Drools is one of the components that will be embedded in (typically) a Java application,
- if your rules are to be edited by the programmers consider first different scenarios like e.g. inhouse engine built on scripting languages in logic tier, or PL/SQL in storage tier; languages are commonly known or easy to learn, the architecture can be easily adopted to your problems; of course, you will still need to develop the QA, release lifecycle etc. on your own.
Do you need to implement the rules to be developed by business rather than the technical staff? BRs are still the way to go, however consider that you need to introduce additional level of abstraction like spreadsheets or DSL. Also check if the BRMS provided fits your organizational need. Sum up the time needed to customize BRMS, hiding rule language complexity and verify if you cannot introduce in-house in-point solution.
Keep in mind that generally a rule based approach is very reasonable to assure business agility, however some specific products like Drools have their own individual characteristics that may not be suitable for your organization. You may still use business rules approach in your system design, but without using any market products if they do not fit your need.