Lessons Learned from Migrating BEM 7.4 to BPPM 9.0 - Part 9: Cell extension and architecture

In BEM 7.4, we made extensive customization in MRL rules to standardize event processing regardless of event sources. Because our events come from 7 different event sources (BMC PATROL, BMC Portal, and 5 other vendors' monitoring tools), we didn't want to write 7 different sets of rules to process events. We wanted all events share the same processing rules as much as possible.

In our standardized event processing rules, each event goes through the following stages: mapping, conversion, filtering, host/device look-up, action look-up, blackout look-up, aggregation/correlation, update, email notification, ticketing, action, and forwarding. Only at mapping and conversion stages, events from different event sources have their own processing rules. All events share the same processing rules starting from filtering stage. This has allowed us to quickly integrate events from any event source into BEM/BPPM cells in a matter of days or even hours.

The advantage of using cell policies is that you don't have to know MRL programming. But policies slow down the cell processing speed a lot. And, most importantly, there is no policy equivalent for execute rule. While we have already made extensive customization in MRL rules, there is no advantage for us to use policies. We disabled all out-of-box policies. We also enforced strict name convention to make our rules easy to support and upgrade. We have about 30 custom rule files supporting over 20 advanced features.

Our cells were architected in three levels. The first level is for look-up. Each event source has its own first level cell so that if one event source is having an event storm it won't affect the events from other event sources. The second level is for update and notification. All event sources share the same second level cell so that events can be correlated easily. The third level is for service impact.

During our migration from BEM 7.4 to BPPM 9.0, we migrated our first-level and second-level cells as BPPM 9.0 remote cells located on their own servers as I talked about it in Part 3. All our custom MRL rules were migrated into the new cells with little change since our customization was kept in separate files. The embedded cell on BPPM server will replace our old third-level service impact cell. Since we did little work for service impact in BEM 7.4, we plan to do a new implementation for service impact once our Atrium upgrade is completed.

The only major change we had to make is the custom GUI display. In BEM 7.4, we made several display templates for administrators, developers, and service desk operators in its Java GUI (BMC Impact Explorer). Since there is no direct migration path from Java GUI to web GUI, we had to re-create all templates in BPPM 9.0 web GUI.