Friday, March 27, 2009

OpenNMS monitor disk space usage by SNMP

#==Overview
This post demonstrates two ways to monitor disk space usage by 2 different SNMP MIBS
1) .iso.org.dod.internet.private.enterprises.ucdavis.dskTable
2) .iso.org.dod.internet.mgmt.mib-2.host.hrStorage


What is the diffrence? Option #1 requires disk path to be hardcoded in snmpd.conf at target system, But Option #2 can monitor all partions by default, even there is need to monitor specific partions, the filter is set on OpenNMS, not target system, So Option #2 is more flexible

The alarm is triggered by threshold in SNMP, so you don't need to setup monitors to trigger alarm. Firstly, Please make sure you have basic snmp working, refer to my post Set up Net-snmp on CentOS

#==ENV
OpenNMS 1.6.2 + Centos 5.2 +net-snmp 5.3.1


#==(1) Monitor disk space usage by dskTable MIB
add the parttiton to be monitored to snmpd.conf

$vi /etc/snmp/snmpd.conf
disk /opt2

#====Test by snmpwalk first


$snmpwalk -v2c 172.16.31.3 -c public .iso.org.dod.internet.private.enterprises.ucdavis.dskTable
UCD-SNMP-MIB::dskPath.1 = STRING: /opt2 

OpenNMS 1.6.2 has set default threshold, so you don't need to config any thing in openNMS. doublecheck the threshold by
GUI->Admin->Manage Thresholds->netsnmp

#====Sample alarm appeared
High threshold exceeded for SNMP datasource ns-dskPercent on interface 172.16.31.3, parms: ds="ns-dskPercent" value="100.0" threshold="90.0" trigger="2" rearm="75.0" label="/opt2" ifIndex="2

#== (2) Monitor disk space usage by hrStorage MIB
no need to add disk path to snmpd.conf, but openNMS needs to be customized.

#===Test by snmpwalk first


$snmpwalk -v2c 172.16.31.3 -c public .iso.org.dod.internet.mgmt.mib-2.host.hrStorage


#==== Find systemOID
This OID is the same to all Net-SNMP agent


$snmptranslate .iso.org.dod.internet.private.enterprises.netSnmp.netSnmpEnumerations.netSnmpAgentOIDs -On

.1.3.6.1.4.1.8072.3.2

#==== Include the SysOID to ./etc/datacollection-config.xml
By default, mib2-host-resources-storage is not included for Net-SNMP
The value in sysoidMask should include your systemOID
For example sysoidMask .1.3.6.1.4.1.8072.3. includes .1.3.6.1.4.1.8072.3.2


systemDef name="Net-SNMP"
sysoidMask.1.3.6.1.4.1.8072.3./sysoidMask
collect
...
includeGroup mib2-host-resources-storage/includeGroup

#====Include your sysOID to ./etc/threshd-configuration.xml



package name="hrstorage"
filterIPADDR != '0.0.0.0' & (nodeSysOID LIKE '.1.3.6.1.4.1.8072.3.2.%' nodeSysOID LIKE '.1.3.6.1.4.1.311.%' nodeSysOID LIKE '.1.3.6.1.4.1.2.3.1.2.1.1.3.%')/filter
..
/package


OpenNMS 1.6.2 has set default threshold, doublecheck the threshold by
GUI->Admin->Manage Thresholds->hrStorage



By default, it monitors all partions. You can create filter to monitor specific partions only

#===Sample alarm appeared
High threshold exceeded for SNMP datasource hrStorageUsed / hrStorageSize * 100.0 on interface 172.16.31.3, parms: ds="hrStorageUsed / hrStorageSize * 100.0" value="94.88754412506304" threshold="90.0" trigger="2" rearm="75.0" label="/opt2" ifIndex="2"

4 comments:

  1. my data-collection-config.xml has:
    "sysoidMask.1.3.6.1.4.1.8072.3.sysoidMask
    collect"
    "includeGroup mib2-host-resource-storage/includeGroup"

    my threshd-configuration.xml file has:

    "package name="hrstorage"
    filter>IPADDR != '0.0.0.0' & (nodeSysOID LIKE '.1.3.6.1.4.1.8072.3.2.%' nodeSysOID LIKE '.1.3.6.1.4.1.311.%' | nodeSysOID LIKE '.1.3.6.1.4.1.2.3.1.2.1.1.3.%')/filter"
    include-range begin="192.168.115.100" end="192.168.115.102"/

    " service name="SNMP" interval="300000" user-defined="false" status="on""
    "parameter key="thresholding-group" value="hrstorage"/
    /service
    /package"


    Finally in the GUI expression-based Thresholds:

    high hrStorageUsed / hrStorageSize * 100.0 hrStorageIndex hrStorageDescr 60.0 75.0 2

    I see my 192.168.115.100 machine has 67% of its root is used. I dont see any alarm? Can you pl point me what iam missing?
    NOTE: I removed the HTML tages as - I am not being allowed to post this comment with the tage.
    Thank you for the post, appreciate it.
    Ven

    ReplyDelete
  2. 1)Verify snmp by snmpget cmd to query partition OID
    2)Turn on DEBUG for Collectd in $OPENNMS_HOME/etc/log4j.properties

    ReplyDelete
  3. Below is Collectd section from log4j.properties, where should I specify DEBUG?
    Also appreciate if you can give me complete command for snmpget.
    Sorry for asking you to spoonfeed, I am new to this.

    # Collectd
    log4j.category.OpenNMS.sCollectd=WARN, COLLECTD
    log4j.additivity.OpenNMS.Collectd=false
    log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
    log4j.appender.COLLECTD.MaxFileSize=100MB
    log4j.appender.COLLECTD.MaxBackupIndex=4
    log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log
    log4j.appender.COLLECTD.layout=org.apache.log4j.PatternLayout
    log4j.appender.COLLECTD.layout.ConversionPattern=%d %-5p [%t] %c{1}: %m%n

    ReplyDelete