The Ten Most Common Mistakes Made By Corporate Adopters of Business Rule Management Systems (BRMS)
~ And How to Avoid Them (Part 1)
by Jan Purchase
The increasing need for agility, transparency, and control of business policy is coercing businesses to extract their business logic from obfuscated code in their IT components and to manage that business logic as a separate, independently-evolving asset, defined in business terms. The appeal of using Business Rule Management Systems (BRMS) to achieve this is clear: when used correctly, these tools can reduce policy change release cycles from weeks to hours and put business policy experts — not IT departments — in the driving seat. In addition, business rules can be used to prevent loss of intellectual capital when staff leave. Business rules offer the ability to federate partial control of business logic to third parties, providing a new way of doing business.
Why then are so many adopters of this established technology jeopardizing their own efforts and reducing the return on their investment by making the same ten mistakes? If you plan to adopt BRMS, how should you avoid doing the same? In the first part of this two-part article, I discuss five common BRMS adoption mistakes I have witnessed over six years of mentoring BRMS pilots in the investment banking and insurance sector.
10. Performance Anxiety
Business rules are often used in high-volume decision processes in which performance is critical. Therefore, it is entirely reasonable that adopters of the technology should be wary about the performance of any BRMS product. However, many organisations allow the introduction of a BRMS into their architecture to distract them from the big picture. This leads them to err in three ways: to assume that any performance issue encountered during the project is solely due to the (newly introduced) BRMS, to be obsessive about rule performance to the extent that other architectural flaws are overlooked, and to test performance in ill-conceived ways.
If a company adopts a BRMS product, surely any performance issues they experience from now on are the fault of the new product? Surely representing business logic in a natural language is less efficient than Java or C#? Well, almost always, the answer is no. Most BRMSs are very efficient software components that, due to rule-oriented optimization techniques, offer better performance than third-generation programming languages. By now, the market has eradicated products with serious performance impediments. In fact, performance issues are most often caused by:
- misuse due to inexperience (e.g., the application of RETE algorithms where it is not appropriate, database access during rule execution),
- mistakes in configuration (e.g., insufficient memory, low SAN bandwidth, or a poor integration), or
- 'knock-on' effects that are not directly attributable to the product (e.g., inefficient caching of reference data).
There is rarely an inherent shortcoming with the tool. Worse, this distraction with BRMS performance can lead to a disproportionate focus on 'optimizing the rules' at any cost, thereby obtaining a small benefit in performance by compromising the business clarity of rules, project budgets, or some other critical success factor.
For example: a recent client achieved a four-fold increase in their rule performance after two man-months effort and was delighted. However, the rule execution was less than 2% of their end-to-end run time. Consequently their effort (an estimated 5% of their total budget) had only bought them a 1.5% improvement in end-to-end performance [((4-1)/4)*2%] — a poor ROI by anyone's standards.
- Ensure your drive to optimize performance is done sufficiently early to allow for radical change if indicated, but not so early that you waste resources on ineffective optimization (i.e., optimization of a non-representative subset of your architecture, or that is later undermined by incremental evolution of your design).
- Ensure you engage personnel with sufficient hands-on experience of the BRMS you have adopted. Systems integrators often lack these resources, despite claims to the contrary. Budget for a few man-weeks of help from a BRMS specialist that has independent certification.
- Ensure you evaluate your performance optimization targets in the context of the entire end-to-end. Check the value of such optimization before committing time to it. Remember: demonstrating linear scalability of processing (with extra CPUs and memory) is often more valuable than shortening the end-to-end processing times of individual transactions.
- If very high data throughput is a feature of your business domain, consider the use of CEP technologies as an alternative to BRMS.
9. Not Working Your Vendor Hard Enough
The market for BRMS (particularly in the US and Europe) is ever more competitive because:
- rule-based approaches are now mainstream (thus clients are more discerning),
- there are more second- and third-generation products on the market, and
- the global belt-tightening brought on by the credit crunch is causing everyone to demand more for less.
This means that BRMS vendors are even more desperate to get your business, especially if it is a large, multi-part, or a key-niche project (e.g., financial services, healthcare, or insurance). It is surprising, therefore, how many clients fail to capitalize on this, both by extracting every benefit possible during the contract negotiation stage itself and, more subtly, by being very demanding in the competitive, multi-vendor 'play-off' that precedes tool selection.
|Be sure you maximise the free benefits that the vendor can provide before negotiation starts. The most effective means of doing this is during tool selection. Use the escalating, competitive, 'proof of concept' approach. After the first round of RFPs and interviews, short list your best 3 - 4 vendors and invite them to 'play off', in a competitive proof of concept (PoC). The goal of this PoC is to solve one (small) problem of your business, or to demonstrate performance in a realistic microcosm of the ultimate system. It works best if the vendors do not know the identity of the competition; some vendors will withdraw if placed against certain competitors. In addition, some may withdraw if the play-off requires a lot of investment on their part; this should prevent you from wasting your time with vendors who are bidding outside of their core niche.
This 'play-off' looks like a straightforward PoC stage and has all of the attendant advantages (enhancing the perceived benefits of BRMS to your business sponsors and making your evaluation exercise effective). However, as the PoC progresses, you point out to each vendor that a rival has demonstrated a key benefit that you would like to see replicated across all competitors. By this means you 'up the ante' of the PoC, thus gaining more free development from the vendors and lowering the project risk for your company. If the vendors get rebellious, offer to pay a percentage of the development costs of the winner. If you run the PoC correctly, the vendor will be compelled to compete or lose.
Ensure your staff is involved in the play-off exercise, so that they can benefit from exposure to the tools under real (stress) conditions. You may have to insist that the proof of concept is developed on your premises to permit this. This strategy also exposes any weaknesses in the product set that you should be aware of before you commit.
Your vendors' responsibility does not end as soon as you select his product. Ensure you continue to use your vendor's advice and resources throughout the project when needed — your success is ultimately their success, don't let them forget it.
8. Unsure Offshore
Although the off-shoring of 3GL development projects (critically, Java and C#) has had mixed success, the off-shoring of (business-oriented) rule development is still a recipe for disappointment. This is largely because of a lack of offshore business skills and BRMS experience. Worse still, some offshore vendors are willing to be completely dishonest about the competencies of their teams, especially when inexperienced clients are concerned. There is a wealth of niche technologies that are utilized in enterprise applications, and not every offshore team can be trained in all of them. The temptation is to cover the most commonly-used ones and bluff about the rest — including BRMS. You should avoid your project being used as the offshore developers training opportunity ... unless, of course, it is priced on that basis and you have mitigated the risk.
For example: two offshore development houses, with which the author has engaged in the past two years, each claimed many man-years experience with BRMS products. When the first was asked to submit a prototype as proof of their expertise, they submitted a tutorial demo provided with the product. When interviewed about their demo, they were unable to alter its behaviour or explain its operation in detail. The second submitted personnel who subsequently admitted that their sole experience was a week 'cramming' on BRMS technology, prior to their involvement on the project.
We don't recommend an offshore approach to BRMS adoption, unless:
- You are very familiar with, and impressed by, the technical ability and honesty of your offshore partners.
- You are using an offshore partner with proven expertise of (and experience with) BRMS.
- You interview all offshore personnel rigorously and lock in named, competent personnel for the duration of your project.
- The offshore team demonstrate their competence by developing a free, working prototype rule application.
- The (BRMS) vendor takes the offshore risk completely.
7. Deployment by Stealth — Subversion of Release Controls using BRMS
Some clients see the enhanced agility of a BRMS as a means of short-circuiting local controls (governing the deployment of code changes) imposed by their own governance bodies. Most organisations, quite sensibly, have restrictions in place enforcing the need to fully unit test, package, and regression test every code change that is deployed into a production environment. This policy frequently reduces production incidents and botched deliveries, at the cost of slowing the deployment cycle and forbidding last minute hot fixing of Jar files or CLR assemblies — a tradeoff that is onerous to some and, occasionally, unnecessarily restrictive.
However, in these organisations, last minute changes to the configuration of a system (typically driven by a text 'properties' file) — as opposed to system code changes — are often permitted with minimal testing. Therefore, once the business policy is expressed as rules, rather than code, the rules are officially labelled 'configuration', thereby permitting fundamental last minute changes under the banner of 'reconfiguration'. Many BRMSs represent the rule repository as an interpreted file, or database, which supports the appearance of configuration to the inexperienced. However, this is misleading; although most BRMSs offer a much more controlled environment for defining a policy than any programming language, to equate this with a reduced need to test is reckless and ultimately self defeating.
The agility that BRMS promises cannot be safely achieved if testing is reduced or circumvented.
Have your cake and eat it: Define a turn-key, automated regression testing suite with test cases directly driven from your business requirements. Then, ensure that all rule bases labelled for deployment are submitted to this system automatically and that deployment continues only if all the tests pass. By this means the deployment of changes can be reduced to hours or even minutes, without any appreciable rise in production incidents, providing your test base is adequately defined, of course. The investment needed to create and maintain this regression test suite is considerable, but the return on the investment is substantial. It enables you to be truly agile with confidence.
Consider this — which is the more valuable resource: developer time, or system downtime due to a production incident?
6. Lack of IT Involvement
Some inexperienced clients are lured, by the optimistic sales copy of BRMS vendors, into believing that business users can adopt BRMS and develop rules with little need for IT practitioners. After all, isn't this the value proposition offered by BRMSs: control of business policy by the business? Proponents of this approach perceive the entry threshold to the BRMS user community as very low; one only needs to understand one's business and express policies in natural language. The vendor's IT consultants will analyse the object model, plumb in the BRMS technology, perhaps write a few sample rules, and the business team can take it from there.
However, without the sustained involvement of an IT team, most BRMS projects are doomed to be of very limited success. There are four main reasons for this:
- Technical Training. Typical business users, unless they have prior BRMS experience, cannot make full use of the tools without extensive training and a period of mentoring by a rule expert. Without both, much of the first few months work of the business rule authors will need to be rewritten or discarded later. Many business users are quite unprepared for the level of precision required for a rule-based representation of their domain. Nor are they ready for the wealth of important choices they must make, in advance, about organising their rule hierarchy — choices that later effect its scalability.
- Debugging. Some tools may claim to make rules completely business oriented, but this ends abruptly when executable rules are first tested. Understanding and fixing rule redundancy, overlaps, subsumption (when one rule effectively subsumes one or more others), incompleteness (when there are situations not covered by any rules), cycles, contradictions, and other errors requires some expertise and understanding of mathematical logic. Debugging the subtleties of RETE ordering of rule execution takes technical know-how.
- Maintenance and Evolution. Considerable technical set up is required to 'plumb' a BRMS into institutional infrastructure and its data architecture. The use of XML may lessen these dependencies, but does not change the fact that most BRMSs require (at least a first draft of) an underlying object model on which to base the rules. Even those that do not will require an integration to occur, once an object model is created by the first few rule set drafts. In addition, as rules evolve, this business object model will require occasional refactoring. The integration between the rules, the model, and the wider infrastructure will also require maintenance.
- Version and Change Control. Dovetailing the internal version control mechanisms and development workflow of a BRMS with the source code control of related artefacts outside the rule engine (on which the former are dependent) will require some custodian outside of the business team.
BRMS are often sold on the premise that they reduce the turnaround time for changes in business policy by cutting out the middle man (i.e., developers). Does this obviate the need for developers? No. BRMSs facilitate an equal partnership between the IT development team (which harvests existing rules from applications, submits them for review, develops a object model that supports all the salient business concepts, interfaces this model to the company's infrastructure, and maintains the model by integrating it with new rules) and the business team (which reviews, amends, and takes ownership of business policy, as expressed in rules, and controls future changes). Figure 1 shows the role of IT developers in the evolution of rules (in purple). The dotted black arrows denote change stimuli that trigger actions:
- Infrastructure to IT, when a change to the infrastructure must be integrated with the existing rule base, and
- Business SME to IT, when a new rule function is required (perhaps access to new data or a new rule consequence).
Figure 1. Role of IT Developers in Rule Evolution
The agility of BRMS is achieved by removing the need for IT development to support every business policy change (by decoupling of the purple and green processes in Figure 1), but it does not obviate the need for IT development teams entirely.
Never forget: Business rule development is still software development. Expect, and plan for, your adoption of rules to require IT involvement and support from the outset. IT will be needed to:
Don't be tempted to use the vendor's IT team instead of growing your own; this will lead to an expensive dependency. Aim to remove your dependency on the vendor's team, incrementally, by the second or third iteration of any major project (i.e., within the first 60%). Have them run the first iteration, collaborate on the second, and coach the third. Ensure that all technological knowledge is transferred to your employees during this time.
• integrate the BRMS and maintain this integration over time,
• help maintain the business object model,
• support necessary business evolution and change control, and
• support regression testing frameworks.
We have seen that BRMS adopters are prone to obsession with the technological details of their projects, or neglect them in favour of the business hype. A balanced view is required, backed by experienced personnel.
In the next part of this article, we explore what happens when the importance of business input and processes is overlooked, as we address the five most common and debilitating mistakes made by adopters of BRMS technology.
|standard citation for this article:
|Jan Purchase, "The Ten Most Common Mistakes Made By Corporate Adopters of Business Rule Management Systems (BRMS) ~ And How to Avoid Them (Part 1)," Business Rules Journal, Vol. 11,
No. 5 (May 2010), URL: http://www.BRCommunity.com/a2010/b535.html
Rules that Give You Too Much Freedom
Use the Right Tool for your Job
Learn from the Expert (Part 3): Get Organized in your Rule Thinking
Learn from the Expert (Part 2) — Textual rules: Out of fashion or a Classic?
Learn from the Expert (Part 1) — A Business Analyst must ask "Why?"
Count your Rules!
The Ten Most Common Mistakes Made By Corporate Adopters of Business Rule Management Systems (BRMS) And How to Avoid Them (Part 2)
Guest Column By Jan Purchase
The Ten Most Common Mistakes Made By Corporate Adopters of Business Rule Management Systems (BRMS) And How to Avoid Them (Part 1)
Guest Column By Jan Purchase
How to Deal with Exceptions in Software Support
Exceptions are just 'some more rules'
Rule Authoring Is a Creative Process
What happened to the B and the M of BRM? ... and how the new notion of business rules documentation got introduced
The Liberty of Rules
The Inference Task
Organizing a Set of Rules
Procedural Logic in the Reasoning Process
What about Methods in Rules?
Different Kinds of Rules and How to Write Them Properly
SBVR: Observations from Initial Experiences
Rule History and Versioning (Part 3)
Rule History and Versioning (Part 2)
Rule History and Versioning (Part 1)
Flexibility and Business Rules
A World Without Rules
The Semantic Web and the Business Rules Approach ~ Differences and Consequences
The Semantic Web and the Business Rules Approach ~ Differences and Similarities
Observations on Business Rules in Europe and the U.S.