Event Integration From 3rd-party Monitoring Tools - Part 11: Develop your own adapter through email or iiSDK

What if your 3rd-party monitoring software cannot invoke an OS command to call msend API, and also cannot send a SNMP trap when an alert is raised?  There are two other methods available but you will need to develop your own adapter in order to integrate events from 3rd-party monitoring software into a BPPM cell.

If your 3rd-party monitoring software can send out an email when an alert is raised, you can consider developing your own email adapter.  You can choose a scripting language you are most comfortable for the development.

Step 1: Locate a computer that is capable of receiving emails, e.g., a UNIX machine with 'sendmail' process running.  

Step 2: Read each incoming email out of its inbox. 

Step 3: Parse both email header and email body.  You will need to understand the format of your email because each 3rd-party monitoring software has its own email format and possibly multiple formats depending on how you set it up.

Step 4: Call msend API with information from your parsed email.

Step 5: Follow the steps discussed in msend API from the posts "Event Integration From 3rd-party Monitoring Tools - Part 2, Part3, Part 4, Part 5" to complete your integration.

Please keep in mind that a custom-developed email adapter has no built-in guaranteed delivery mechanism.  You would need to use simple redundancy discussed in SNMP adapter for high availability.

The last but not the least method is to develop your own adapter using the free iiSDK provided by BMC. iiSDK  provides you a library of C or Java APIs that you can call from your C or Java program.  From your C or Java code you will build an executable program that runs as an adapter. 

Since you write your own code, you have more flexibility as what you want your adapter to do.  You can pull events from your 3rd-party monitoring software while you can only push events using msend API, SNMP adapter or email adapter.  You can also send events back to your 3rd-party monitoring software from BPPM cell to create a 2-way integration.  

Unless you are an experienced C or Java developer and also have the resource to maintain such a complex event adapter, I would suggest you to stay away from iiSDK. You won't get much support from BMC because iiSDK is free. 

What if you need a 2-way integration?  Actually the best way to send BPPM events back to your 3rd-party monitoring software (mostly with an incident ticket number and status) is to use the native interface/API offered by your 3rd-party monitoring software.  Comparing to using iiSDK, the native interface/API is faster, more reliable, less overhead, and much easier to maintain.

Please feel free to contact us if you need help for a custom-developed event adapter using either email or iiSDK.  We have extensive custom development experience and can give you a quick turn around for a cost-effective solution.

Event Integration From 3rd-party Monitoring Tools - Part 10: SNMP trap adapter mapping and conversion

In our previous post "Event Integration From 3rd-party Monitoring Tools - Part 5: Msend API Mapping and Conversion", we discussed three steps needed to perform mapping and conversion.  Now since we are using BMC out-of-box SNMP trap adapter, do we still need to follow the same three steps? Let's take a closer look at the same three steps and see how much those steps still apply to SNMP trap adapter.

First step: What BPPM cell classes to map to. Since SNMP trap adapter will automatically map your SNMP traps into event class names based on the MIB file you provided, you don't have to do anything here. The new event classes extracted out of MIB file are defined in these two baroc files generated in SNMP trap adapter configuration step.

Second step: What slots in BPPM cell to map to.  The slot names of those event classes automatically created in step one are also automatically created based on the MIB file.  They are different from the slot names you usually use to make decision on how to process events. For example, if you decide whether or not to create an incident ticket from an event based on the value of its severity slot, you will need to write a MRL rule or policy to set the value of severity slot based on the value of another slot.  If you don't perform mapping in this step, your severity slot will just contain the default value for all your SNMP trap events - something you may not want to see.

Again, the commonly mapped slots include mc_host, mc_object_class, mc_object, mc_tool_class, mc_tool_key, mc_parameter, mc_parameter_value, severity, mc_priority, and msg.  Here mc_tool_key refers to the unique event (alert) ID used in your 3rd-party monitoring software.  It can be used in the de-dup rule mentioned in our last post "Event Integration From 3rd-party Monitoring Tools - Part 9: SNMP Trap Adapter Execution and High Availability"

Third step: How to map.  Similar to the case with msend API, you may map directly by copying the value from one slot to another, or you may need to perform value conversion especially for enumerable types such as severity and priority.  Again you can take advantage of dynamic tables to make value conversion easy to code and easy to maintain.

Event Integration From 3rd-party Monitoring Tools - Part 9: SNMP trap adapter execution and high availability

With BMC SNMP Trap Adapter, most of the work required for execution has been done out of box in mcxa executable and its configuration file etc\mcxa.conf. However, there are a few things specific about SNMP traps that we do want to pay attention to.

First, if your 3rd-party monitoring software allows, filter the SNMP traps to limit the number of traps to what you really need.  SNMP traps are in general very chatty.  Without filtering, you can expect to see large number of SNMP traps sending to your BPPM cell.

Second, pay attention to SNMP trap adapter log file. If you have large traps, the error messages in your log file will tell you to increase the value of parameter SnmpTrapLength in mcxa.conf.  The default value for SnmpTrapLength is 8192 bytes.  You need to double that value for large traps.

Third, implement high availability for SNMP trap adapter.  In general, SNMP traps are not logged, buffered, or re-tried.  You would easily miss some important SNMP traps from a short period of network outages. However since BMC doesn't provide out-of-box high availability for SNMP trap adapter, we would have to go with simple redundancy.

Install and configure two identical instances of SNMP trap adapter on two different servers. If you have an active/standby BPPM cell cluster, you can install a SNMP trap adapter on each server of the cluster.  This is simply a convenience, not a requirement.  Then configure these two SNMP trap adapter severs as two SNMP trap destinations from your 3rd-party monitoring software.

Now for each SNMP trap sent out from your 3rd-party monitoring software, you will receive two events with one event converted from each SNMP trap adapter.  You will need to write a de-dup MRL rule in your BPPM cell to drop one of the events.  Your de-dup rule should compare the original trap IDs to determine if two traps are identical.  If the trap ID is short, meaning that the same ID can be re-used fairly quickly, you will also need to compare the original timestamps as well to determine if two events are truly duplicates.

Event Integration From 3rd-party Monitoring Tools - Part 8: SNMP trap adapter configuration from command line

We discussed how to configure SNMP trap adapter using GUI in the last post. What if you can't find a server with no TOMCAT running to install SNMP Configuration Manager?  You do have an option to configure SNMP trap adapter from command line.

Method 2: From command line

In order to configure SNMP trap adapter from command line, you need to download and install open source net-snmp package from http://www.net-snmp.org/download.html

Once it is installed, define an environment variable SNMP_DIR pointing to net-snmp installation direcotry.  For example:

set SNMP_DIR=C:\SNMPAdapter\ucd-snmp-4.2.3\usr

Then copy your MIB file to %SNMP_DIR%\mibs\ directory.  Run the following command:

mib2map.pl -m ALL -M "%SNMP_DIR%\mibs;%SNMP_DIR%\share\snmp\mibs" -s %SNMP_DIR%\bin\snmptranslate.exe

Where mib2map.pl is a perl script already installed in your %MCELL_HOME%\bin\ directory as part of BMC Event Adapters.

In the current directory, you should see the following 4 files generated as the result of running the above mib2map.pl command:

mcsnmptrapdmib.dat

mcsnmptrapdmib.map

mcsnmptrapdmib.baroc

mcsnmptrapdmibe.baroc

Now you have generated the above 4 files either from GUI or command line, copy both mcsnmptrapdmib.baroc and mcsnmptrapdmibe.baroc to %MCELL_DIR%\etc\<cell_name>\kb\classes\ directory.  Add mcsnmptrapdmibe followed by mcsnmptrapdmib to .load file in %MCELL_DIR%\etc\<cell_name>\kb\classes\ directory.  The <cell_name> here needs to match the value of parameter ServerName in your mxca.conf file.

Copy both mcsnmptrapdmib.dat and mcsnmptrapdmib.map to %MCELL_DIR%\etc\ directory. Rename mcsnmptrapdmib.dat to mcsnmptrapd.dat.  Rename mcsnmptrapdmib.map to mcsnmptrapd.map.

Recompile the cell.  Restart the cell.  Stop and then start SNMP trap adapter after a couple of minutes.  Now you can configure your 3rd-party monitoring software to send SNMP traps to your SNMP trap adapter.  You should see events with class names defined in the MIB file shown up in your cell.

Event Integration From 3rd-party Monitoring Tools - Part 7: SNMP trap adapter configuration using GUI

Since SNMP trap adapter is responsible to translate a SNMP trap into an event in BAROC format, it needs some instruction to know how to translate.  This instruction is contained in the MIB file provided by the vendor of your 3rd-party monitoring software. You need to ask the administrator of your 3rd-party monitoring software to send you a copy of MIB file before you start configuring SNMP trap adapter.

The purpose of configuring SNMP trap adapter is to generate the following 4 files from your MIB file:

mcsnmptrapdmib.dat

mcsnmptrapdmib.map

mcsnmptrapdmib.baroc

mcsnmptrapdmibe.baroc

You can choose either GUI method or command line method to achieve that.  This is an one-time process and you will use the same 4 files for all your development, QA, and production environments.  You only need to go through this configuration process again if your MIB file has changed.

Method 1: Using GUI

Download SNMP Configuration Manager from BMC EPD site.  Install SNMP Configuration Manager. You will be asked to input JDK Home directory, Perl Home directory, and TOMCAT port numbers. The default TOMCAT port numbers are 8080 (listening) and 8005 (shutdown). Verify both ports are available first.

Since this is an one-time process and SNMP Configuration Manager is not needed during run time, I recommend to install SNMP Configuration Manager on a test server that has no TOMCAT running.  It is best not to install SNMP Configuration Manager on the same server where BPPM server is located to avoid TOMCAT port conflict.

Once SNMP Configuration Manager has been installed, you can access it through this URL:

http://<servername>:8080/snmpadapter/

SNMP Configuration Manager will ask you for the location of your MIB file.  As the result of publishing the MIB file, it will generate the above four files. Verify that mcsnmptrapdmib.dat and mcsnmptrapdmib.map are located in %MCELL_HOME%\etc\ directory.  Verify that mcsnmptrapdmib.baroc and mcsnmptrapdmibe.baroc are located in %MCELL_HOME%\bin\ directory.

We will discuss how to configure SNMP trap adapter using command line in the next post.