XML collection not working

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

XML collection not working

BeeJay

Hi all,

 

I have been trying to implement a custom monitor for CUPS printers using the XML collector, but I’m having problems getting it to collect any data.

 

Can someone please confirm that my configuration is correct, or let me know what I’ve missed.

 

We’re running opennms 20.0.1

 

 

 

I have a webserver set up that returns the metrics I care about in xml format:

 

[root@aue-s-opennms01 etc]# curl http://10.1.2.3:8000/printers.xml

<?xml version="1.0" encoding="UTF-8"?>

<printers>

     <printer name="Printer1">

           <njobs>5</njobs>

     </printer>

     <printer name="Printer2">

           <njobs>0</njobs>

     </printer>

     <printer name="Printer3">

           <njobs>0</njobs>

     </printer>

</printers>

 

 

Here is (I think) all the relevant configuration bits:

 

xml-datacollection-config.xml:

 

    <xml-collection name="ABBCS-CUPS">

        <rrd step="300">

            <rra>RRA:AVERAGE:0.5:1:2016</rra>

            <rra>RRA:AVERAGE:0.5:12:1488</rra>

            <rra>RRA:AVERAGE:0.5:288:366</rra>

            <rra>RRA:MAX:0.5:288:366</rra>

            <rra>RRA:MIN:0.5:288:366</rra>

        </rrd>

        <xml-source url="http://{ipaddr}:8000/printers.xml">

            <import-groups>xml-datacollection/CUPS-Printers.xml</import-groups>

        </xml-source>

    </xml-collection>

 

 

xml-datacollection/CUPS-Printers.xml:

 

<xml-groups>

    <xml-group name="CUPS-Printer-stats" resource-type="CUPSPrinters"

                resource-xpath="/printers/printer"

                key-xpath="@name">

      <xml-object name="printerName" type="string" xpath="@name" />

      <xml-object name="njobs" type="GAUGE" xpath="njobs" />

    </xml-group>

</xml-groups>

 

collectd-configuration.xml:

 

   <package name="ABBCS-CUPS">

      <filter>(IPADDR != '0.0.0.0')</filter>

      <include-range begin="1.1.1.1" end="254.254.254.254"/>

      <include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"/>

      <service name="ABBCS-CUPS" interval="300000" user-defined="false" status="on">

         <parameter key="collection" value="ABBCS-CUPS"/>

         <parameter key="handler-class" value="org.opennms.protocols.xml.vtdxml.DefaultVTDXmlCollectionHandler"/>

      </service>

   </package>

 

 

   <collector service="ABBCS-CUPS" class-name="org.opennms.protocols.xml.collector.XmlCollector"/>

 

 

datacollection-configuration.xml:

 

<?xml version="1.0"?>

<datacollection-config rrdRepository="/opt/opennms/share/rrd/snmp/">

    <snmp-collection name="ABBCS" snmpStorageFlag="select">

        <rrd step="300">

            <rra>RRA:AVERAGE:0.5:1:8928</rra>

            <rra>RRA:AVERAGE:0.5:12:8784</rra>

            <rra>RRA:AVERAGE:0.5:480:366</rra>

            <rra>RRA:MIN:0.5:12:8784</rra>

            <rra>RRA:MAX:0.5:12:8784</rra>

        </rrd>

        <include-collection dataCollectionGroup="ABBCS-CUPS"/>

    </snmp-collection>

</datacollection-config>

 

 

 

datacollection/ABBCS-CUPS.xml:

 

<datacollection-group name="ABBCS-CUPS">

  <resourceType name="CUPSPrinters" label="CUPS Printers"

   resourceLabel="Printer ${printerName}" >

     <persistenceSelectorStrategy

          class="org.opennms.netmgt.collection.support.PersistAllSelectorStrategy"/>

     <storageStrategy

          class="org.opennms.netmgt.collection.support.IndexStorageStrategy"/>

  </resourceType>

</datacollection-group>

 

 

poller-configuration.xml:

 

   <package name="ABBCS-CUPS">

      <filter>((IPADDR != '0.0.0.0') &amp; (catincServers))</filter>

      <include-range begin="1.1.1.1" end="254.254.254.254"/>

      <include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"/>

      <rrd step="300">

         <rra>RRA:AVERAGE:0.5:1:2016</rra>

         <rra>RRA:AVERAGE:0.5:12:1488</rra>

         <rra>RRA:AVERAGE:0.5:288:366</rra>

         <rra>RRA:MAX:0.5:288:366</rra>

         <rra>RRA:MIN:0.5:288:366</rra>

      </rrd>

      <service name="ABBCS-CUPS" interval="300000" user-defined="false" status="on">

         <parameter key="retry" value="1"/>

         <parameter key="timeout" value="3000"/>

         <parameter key="port" value="8000"/>

         <parameter key="url" value="/printers.xml"/>

         <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />

         <parameter key="rrd-base-name" value="cups" />

         <parameter key="response" value="200-202,299"/>

         <parameter key="response-text" value=".*printer.*"/>

      </service>

      <downtime begin="0" end="300000" interval="30000"/>

      <downtime begin="300000" end="43200000" interval="300000"/>

      <downtime begin="43200000" end="432000000" interval="600000"/>

      <downtime begin="432000000" delete="true"/>

   </package>

 

 

   <monitor service="ABBCS-CUPS" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor"/>

 

 

Any assistance greatly appreciated.

 

Regards,

John Blackburn.

 

 

 


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: XML collection not working

Ronald Roskens
Does it work if you take out the handler-class parameter in the collectd-configuration.xml? The size of your XML is so small, that I don’t think VTD is going to make that much of a difference in speed for parsing it, so lets try with the default handler first. (You didn’t list which packages you have installed. The VTD handler isn’t installed by default and you need an extra package for it.

On Jul 17, 2017, at 12:32 AM, JohnD Blackburn <[hidden email]> wrote:

Hi all,
 
I have been trying to implement a custom monitor for CUPS printers using the XML collector, but I’m having problems getting it to collect any data.
 
Can someone please confirm that my configuration is correct, or let me know what I’ve missed.
 
We’re running opennms 20.0.1
 
 
 
I have a webserver set up that returns the metrics I care about in xml format:
 
[root@aue-s-opennms01 etc]# curl http://10.1.2.3:8000/printers.xml
<?xml version="1.0" encoding="UTF-8"?>
<printers>
     <printer name="Printer1">
           <njobs>5</njobs>
     </printer>
     <printer name="Printer2">
           <njobs>0</njobs>
     </printer>
     <printer name="Printer3">
           <njobs>0</njobs>
     </printer>
</printers>
 
 
Here is (I think) all the relevant configuration bits:
 
xml-datacollection-config.xml:
 
    <xml-collection name="ABBCS-CUPS">
        <rrd step="300">
            <rra>RRA:AVERAGE:0.5:1:2016</rra>
            <rra>RRA:AVERAGE:0.5:12:1488</rra>
            <rra>RRA:AVERAGE:0.5:288:366</rra>
            <rra>RRA:MAX:0.5:288:366</rra>
            <rra>RRA:MIN:0.5:288:366</rra>
        </rrd>
        <xml-source url="http://{ipaddr}:8000/printers.xml">
            <import-groups>xml-datacollection/CUPS-Printers.xml</import-groups>
        </xml-source>
    </xml-collection>
 
 
xml-datacollection/CUPS-Printers.xml:
 
<xml-groups>
    <xml-group name="CUPS-Printer-stats" resource-type="CUPSPrinters"
                resource-xpath="/printers/printer"
                key-xpath="@name">
      <xml-object name="printerName" type="string" xpath="@name" />
      <xml-object name="njobs" type="GAUGE" xpath="njobs" />
    </xml-group>
</xml-groups>
 
collectd-configuration.xml:
 
   <package name="ABBCS-CUPS">
      <filter>(IPADDR != '0.0.0.0')</filter>
      <include-range begin="1.1.1.1" end="254.254.254.254"/>
      <include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"/>
      <service name="ABBCS-CUPS" interval="300000" user-defined="false" status="on">
         <parameter key="collection" value="ABBCS-CUPS"/>
         <parameter key="handler-class" value="org.opennms.protocols.xml.vtdxml.DefaultVTDXmlCollectionHandler"/>
      </service>
   </package>
 
 
   <collector service="ABBCS-CUPS" class-name="org.opennms.protocols.xml.collector.XmlCollector"/>
 
 
datacollection-configuration.xml:
 
<?xml version="1.0"?>
<datacollection-config rrdRepository="/opt/opennms/share/rrd/snmp/">
    <snmp-collection name="ABBCS" snmpStorageFlag="select">
        <rrd step="300">
            <rra>RRA:AVERAGE:0.5:1:8928</rra>
            <rra>RRA:AVERAGE:0.5:12:8784</rra>
            <rra>RRA:AVERAGE:0.5:480:366</rra>
            <rra>RRA:MIN:0.5:12:8784</rra>
            <rra>RRA:MAX:0.5:12:8784</rra>
        </rrd>
        <include-collection dataCollectionGroup="ABBCS-CUPS"/>
    </snmp-collection>
</datacollection-config>
 
 
 
datacollection/ABBCS-CUPS.xml:
 
<datacollection-group name="ABBCS-CUPS">
  <resourceType name="CUPSPrinters" label="CUPS Printers"
   resourceLabel="Printer ${printerName}" >
     <persistenceSelectorStrategy
          class="org.opennms.netmgt.collection.support.PersistAllSelectorStrategy"/>
     <storageStrategy
          class="org.opennms.netmgt.collection.support.IndexStorageStrategy"/>
  </resourceType>
</datacollection-group>
 
 
poller-configuration.xml:
 
   <package name="ABBCS-CUPS">
      <filter>((IPADDR != '0.0.0.0') &amp; (catincServers))</filter>
      <include-range begin="1.1.1.1" end="254.254.254.254"/>
      <include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"/>
      <rrd step="300">
         <rra>RRA:AVERAGE:0.5:1:2016</rra>
         <rra>RRA:AVERAGE:0.5:12:1488</rra>
         <rra>RRA:AVERAGE:0.5:288:366</rra>
         <rra>RRA:MAX:0.5:288:366</rra>
         <rra>RRA:MIN:0.5:288:366</rra>
      </rrd>
      <service name="ABBCS-CUPS" interval="300000" user-defined="false" status="on">
         <parameter key="retry" value="1"/>
         <parameter key="timeout" value="3000"/>
         <parameter key="port" value="8000"/>
         <parameter key="url" value="/printers.xml"/>
         <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
         <parameter key="rrd-base-name" value="cups" />
         <parameter key="response" value="200-202,299"/>
         <parameter key="response-text" value=".*printer.*"/>
      </service>
      <downtime begin="0" end="300000" interval="30000"/>
      <downtime begin="300000" end="43200000" interval="300000"/>
      <downtime begin="43200000" end="432000000" interval="600000"/>
      <downtime begin="432000000" delete="true"/>
   </package>
 
 
   <monitor service="ABBCS-CUPS" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor"/>
 
 
Any assistance greatly appreciated.
 
Regards,
John Blackburn.
 
 
 
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: XML collection not working

BeeJay

Thankyou for your response.

 

I installed the VTD required packages before I tried setting it as a handler.

 

As suggested, I’ve  removed the handler class entry from collectd-configuration.xml, but it was still not working.

 

I have just now added the following to the default-foreign-source.xml

 

      <detector name="ABBCS-CUPS" class="org.opennms.netmgt.provision.detector.simple.HttpDetector">

         <parameter key="url" value="/printers.xml"/>

         <parameter key="port" value="8000"/>

      </detector>

 

Which seems to have fixed the issue, but then I found I also needed to add a “~” to the “response-text” parameter in poller-configuration.xml:

 

         <parameter key="response-text" value="~.*printer.*"/>

 

 

The documentation for the XML collector doesn’t mention default-foreign-source.xml at all, so is this something that I shouldn’t really need, and I’ve missed something else in my configuration, or is this  just an oversight, and should be in the doco?

 

 

 

 


                                                                                                       

 

 

ABB logo

John Blackburn

 

Solution Architect - Cloud Services

 

Enterprise Software

 

 

 

Level 1, 757 Ann Street

 

Fortitude Valley, Queensland, 4006

AUSTRALIA

Phone: +61 7 33033219

 

Mobile: +61 403 344 524

E-Mail:  [hidden email]

www.abb.com

 

 

 

From: Ronald Roskens [mailto:[hidden email]]
Sent: Tuesday, 18 July 2017 11:31 PM
To: General OpenNMS Discussion <[hidden email]>
Subject: Re: [opennms-discuss] XML collection not working

 

Does it work if you take out the handler-class parameter in the collectd-configuration.xml? The size of your XML is so small, that I don’t think VTD is going to make that much of a difference in speed for parsing it, so lets try with the default handler first. (You didn’t list which packages you have installed. The VTD handler isn’t installed by default and you need an extra package for it.

 

On Jul 17, 2017, at 12:32 AM, JohnD Blackburn <[hidden email]> wrote:

 

Hi all,

 

I have been trying to implement a custom monitor for CUPS printers using the XML collector, but I’m having problems getting it to collect any data.

 

Can someone please confirm that my configuration is correct, or let me know what I’ve missed.

 

We’re running opennms 20.0.1

 

 

 

I have a webserver set up that returns the metrics I care about in xml format:

 

[root@aue-s-opennms01 etc]# curl http://10.1.2.3:8000/printers.xml

<?xml version="1.0" encoding="UTF-8"?>

<printers>

     <printer name="Printer1">

           <njobs>5</njobs>

     </printer>

     <printer name="Printer2">

           <njobs>0</njobs>

     </printer>

     <printer name="Printer3">

           <njobs>0</njobs>

     </printer>

</printers>

 

 

Here is (I think) all the relevant configuration bits:

 

xml-datacollection-config.xml:

 

    <xml-collection name="ABBCS-CUPS">

        <rrd step="300">

            <rra>RRA:AVERAGE:0.5:1:2016</rra>

            <rra>RRA:AVERAGE:0.5:12:1488</rra>

            <rra>RRA:AVERAGE:0.5:288:366</rra>

            <rra>RRA:MAX:0.5:288:366</rra>

            <rra>RRA:MIN:0.5:288:366</rra>

        </rrd>

        <xml-source url="http://{ipaddr}:8000/printers.xml">

            <import-groups>xml-datacollection/CUPS-Printers.xml</import-groups>

        </xml-source>

    </xml-collection>

 

 

xml-datacollection/CUPS-Printers.xml:

 

<xml-groups>

    <xml-group name="CUPS-Printer-stats" resource-type="CUPSPrinters"

                resource-xpath="/printers/printer"

                key-xpath="@name">

      <xml-object name="printerName" type="string" xpath="@name" />

      <xml-object name="njobs" type="GAUGE" xpath="njobs" />

    </xml-group>

</xml-groups>

 

collectd-configuration.xml:

 

   <package name="ABBCS-CUPS">

      <filter>(IPADDR != '0.0.0.0')</filter>

      <include-range begin="1.1.1.1" end="254.254.254.254"/>

      <include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"/>

      <service name="ABBCS-CUPS" interval="300000" user-defined="false" status="on">

         <parameter key="collection" value="ABBCS-CUPS"/>

         <parameter key="handler-class" value="org.opennms.protocols.xml.vtdxml.DefaultVTDXmlCollectionHandler"/>

      </service>

   </package>

 

 

   <collector service="ABBCS-CUPS" class-name="org.opennms.protocols.xml.collector.XmlCollector"/>

 

 

datacollection-configuration.xml:

 

<?xml version="1.0"?>

<datacollection-config rrdRepository="/opt/opennms/share/rrd/snmp/">

    <snmp-collection name="ABBCS" snmpStorageFlag="select">

        <rrd step="300">

            <rra>RRA:AVERAGE:0.5:1:8928</rra>

            <rra>RRA:AVERAGE:0.5:12:8784</rra>

            <rra>RRA:AVERAGE:0.5:480:366</rra>

            <rra>RRA:MIN:0.5:12:8784</rra>

            <rra>RRA:MAX:0.5:12:8784</rra>

        </rrd>

        <include-collection dataCollectionGroup="ABBCS-CUPS"/>

    </snmp-collection>

</datacollection-config>

 

 

 

datacollection/ABBCS-CUPS.xml:

 

<datacollection-group name="ABBCS-CUPS">

  <resourceType name="CUPSPrinters" label="CUPS Printers"

   resourceLabel="Printer ${printerName}" >

     <persistenceSelectorStrategy

          class="org.opennms.netmgt.collection.support.PersistAllSelectorStrategy"/>

     <storageStrategy

          class="org.opennms.netmgt.collection.support.IndexStorageStrategy"/>

  </resourceType>

</datacollection-group>

 

 

poller-configuration.xml:

 

   <package name="ABBCS-CUPS">

      <filter>((IPADDR != '0.0.0.0') &amp; (catincServers))</filter>

      <include-range begin="1.1.1.1" end="254.254.254.254"/>

      <include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"/>

      <rrd step="300">

         <rra>RRA:AVERAGE:0.5:1:2016</rra>

         <rra>RRA:AVERAGE:0.5:12:1488</rra>

         <rra>RRA:AVERAGE:0.5:288:366</rra>

         <rra>RRA:MAX:0.5:288:366</rra>

         <rra>RRA:MIN:0.5:288:366</rra>

      </rrd>

      <service name="ABBCS-CUPS" interval="300000" user-defined="false" status="on">

         <parameter key="retry" value="1"/>

         <parameter key="timeout" value="3000"/>

         <parameter key="port" value="8000"/>

         <parameter key="url" value="/printers.xml"/>

         <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />

         <parameter key="rrd-base-name" value="cups" />

         <parameter key="response" value="200-202,299"/>

         <parameter key="response-text" value=".*printer.*"/>

      </service>

      <downtime begin="0" end="300000" interval="30000"/>

      <downtime begin="300000" end="43200000" interval="300000"/>

      <downtime begin="43200000" end="432000000" interval="600000"/>

      <downtime begin="432000000" delete="true"/>

   </package>

 

 

   <monitor service="ABBCS-CUPS" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor"/>

 

 

Any assistance greatly appreciated.

 

Regards,

John Blackburn.

 

 

 

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, 
Slashdot.org! http://sdm.link/slashdot_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: XML collection not working

Ronald Roskens
I can understand the confusion. Services need to be discovered (either manually or automatically) first before you can either poll them or collect data from them.

What documentation or tutorials have you read through so far? I’d like to know what was missed so that this can be improved for the next person who tries to do this.

Ron

On Jul 18, 2017, at 10:09 PM, JohnD Blackburn <[hidden email]> wrote:

Thankyou for your response.
 
I installed the VTD required packages before I tried setting it as a handler.
 
As suggested, I’ve  removed the handler class entry from collectd-configuration.xml, but it was still not working.
 
I have just now added the following to the default-foreign-source.xml
 
      <detector name="ABBCS-CUPS" class="org.opennms.netmgt.provision.detector.simple.HttpDetector">
         <parameter key="url" value="/printers.xml"/>
         <parameter key="port" value="8000"/>
      </detector>
 
Which seems to have fixed the issue, but then I found I also needed to add a “~” to the “response-text” parameter in poller-configuration.xml:
 
         <parameter key="response-text" value="~.*printer.*"/>
 
 
The documentation for the XML collector doesn’t mention default-foreign-source.xml at all, so is this something that I shouldn’t really need, and I’ve missed something else in my configuration, or is this  just an oversight, and should be in the doco?
 
 
 
 

                                                                                                       
 
 
<image001.gif>
John Blackburn

 

Solution Architect - Cloud Services

 

Enterprise Software

 

 
 
Level 1, 757 Ann Street

 

Fortitude Valley, Queensland, 4006
AUSTRALIA
Phone: +61 7 33033219

 

Mobile: +61 403 344 524
E-Mail:  [hidden email]

 

 
 
From: Ronald Roskens [[hidden email]] 
Sent: Tuesday, 18 July 2017 11:31 PM
To: General OpenNMS Discussion <[hidden email]>
Subject: Re: [opennms-discuss] XML collection not working
 
Does it work if you take out the handler-class parameter in the collectd-configuration.xml? The size of your XML is so small, that I don’t think VTD is going to make that much of a difference in speed for parsing it, so lets try with the default handler first. (You didn’t list which packages you have installed. The VTD handler isn’t installed by default and you need an extra package for it.
 
On Jul 17, 2017, at 12:32 AM, JohnD Blackburn <[hidden email]> wrote:
 
Hi all,
 
I have been trying to implement a custom monitor for CUPS printers using the XML collector, but I’m having problems getting it to collect any data.
 
Can someone please confirm that my configuration is correct, or let me know what I’ve missed.
 
We’re running opennms 20.0.1
 
 
 
I have a webserver set up that returns the metrics I care about in xml format:
 
[root@aue-s-opennms01 etc]# curl http://10.1.2.3:8000/printers.xml
<?xml version="1.0" encoding="UTF-8"?>
<printers>
     <printer name="Printer1">
           <njobs>5</njobs>
     </printer>
     <printer name="Printer2">
           <njobs>0</njobs>
     </printer>
     <printer name="Printer3">
           <njobs>0</njobs>
     </printer>
</printers>
 
 
Here is (I think) all the relevant configuration bits:
 
xml-datacollection-config.xml:
 
    <xml-collection name="ABBCS-CUPS">
        <rrd step="300">
            <rra>RRA:AVERAGE:0.5:1:2016</rra>
            <rra>RRA:AVERAGE:0.5:12:1488</rra>
            <rra>RRA:AVERAGE:0.5:288:366</rra>
            <rra>RRA:MAX:0.5:288:366</rra>
            <rra>RRA:MIN:0.5:288:366</rra>
        </rrd>
        <xml-source url="<a href="http://{ipaddr}:8000/printers.xml" style="color: purple; text-decoration: underline;" class="">http://{ipaddr}:8000/printers.xml">
            <import-groups>xml-datacollection/CUPS-Printers.xml</import-groups>
        </xml-source>
    </xml-collection>
 
 
xml-datacollection/CUPS-Printers.xml:
 
<xml-groups>
    <xml-group name="CUPS-Printer-stats" resource-type="CUPSPrinters"
                resource-xpath="/printers/printer"
                key-xpath="@name">
      <xml-object name="printerName" type="string" xpath="@name" />
      <xml-object name="njobs" type="GAUGE" xpath="njobs" />
    </xml-group>
</xml-groups>
 
collectd-configuration.xml:
 
   <package name="ABBCS-CUPS">
      <filter>(IPADDR != '0.0.0.0')</filter>
      <include-range begin="1.1.1.1" end="254.254.254.254"/>
      <include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"/>
      <service name="ABBCS-CUPS" interval="300000" user-defined="false" status="on">
         <parameter key="collection" value="ABBCS-CUPS"/>
         <parameter key="handler-class" value="org.opennms.protocols.xml.vtdxml.DefaultVTDXmlCollectionHandler"/>
      </service>
   </package>
 
 
   <collector service="ABBCS-CUPS" class-name="org.opennms.protocols.xml.collector.XmlCollector"/>
 
 
datacollection-configuration.xml:
 
<?xml version="1.0"?>
<datacollection-config rrdRepository="/opt/opennms/share/rrd/snmp/">
    <snmp-collection name="ABBCS" snmpStorageFlag="select">
        <rrd step="300">
            <rra>RRA:AVERAGE:0.5:1:8928</rra>
            <rra>RRA:AVERAGE:0.5:12:8784</rra>
            <rra>RRA:AVERAGE:0.5:480:366</rra>
            <rra>RRA:MIN:0.5:12:8784</rra>
            <rra>RRA:MAX:0.5:12:8784</rra>
        </rrd>
        <include-collection dataCollectionGroup="ABBCS-CUPS"/>
    </snmp-collection>
</datacollection-config>
 
 
 
datacollection/ABBCS-CUPS.xml:
 
<datacollection-group name="ABBCS-CUPS">
  <resourceType name="CUPSPrinters" label="CUPS Printers"
   resourceLabel="Printer ${printerName}" >
     <persistenceSelectorStrategy
          class="org.opennms.netmgt.collection.support.PersistAllSelectorStrategy"/>
     <storageStrategy
          class="org.opennms.netmgt.collection.support.IndexStorageStrategy"/>
  </resourceType>
</datacollection-group>
 
 
poller-configuration.xml:
 
   <package name="ABBCS-CUPS">
      <filter>((IPADDR != '0.0.0.0') &amp; (catincServers))</filter>
      <include-range begin="1.1.1.1" end="254.254.254.254"/>
      <include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"/>
      <rrd step="300">
         <rra>RRA:AVERAGE:0.5:1:2016</rra>
         <rra>RRA:AVERAGE:0.5:12:1488</rra>
         <rra>RRA:AVERAGE:0.5:288:366</rra>
         <rra>RRA:MAX:0.5:288:366</rra>
         <rra>RRA:MIN:0.5:288:366</rra>
      </rrd>
      <service name="ABBCS-CUPS" interval="300000" user-defined="false" status="on">
         <parameter key="retry" value="1"/>
         <parameter key="timeout" value="3000"/>
         <parameter key="port" value="8000"/>
         <parameter key="url" value="/printers.xml"/>
         <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
         <parameter key="rrd-base-name" value="cups" />
         <parameter key="response" value="200-202,299"/>
         <parameter key="response-text" value=".*printer.*"/>
      </service>
      <downtime begin="0" end="300000" interval="30000"/>
      <downtime begin="300000" end="43200000" interval="300000"/>
      <downtime begin="43200000" end="432000000" interval="600000"/>
      <downtime begin="432000000" delete="true"/>
   </package>
 
 
   <monitor service="ABBCS-CUPS" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor"/>
 
 
Any assistance greatly appreciated.
 
Regards,
John Blackburn.
 
 
 
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, 
Slashdot.org! http://sdm.link/slashdot_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
 
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: XML collection not working

BeeJay

The main document I used for setting up the XML collection was this one:

 

https://wiki.opennms.org/wiki/XML_Collector

 

which makes no reference to default foreign source definitions or discovery.

 

I think it needs a section on discovery pointing people to documentation on how to discover the service and/or what methods of discovery would be valid. Also, it is not obvious that XML collection is (potentially) a foreign source so it never occurred to me to search for doco on configuring the default foreign source.

 

Thanks again for your response.

 

Regards,

John

 

 

 


                                                                                                       

 

 

ABB logo

John Blackburn

 

Solution Architect - Cloud Services

 

Enterprise Software

 

 

 

Level 1, 757 Ann Street

 

Fortitude Valley, Queensland, 4006

AUSTRALIA

Phone: +61 7 33033219

 

Mobile: +61 403 344 524

E-Mail:  [hidden email]

www.abb.com

 

 

 

From: Ronald Roskens [mailto:[hidden email]]
Sent: Wednesday, 19 July 2017 11:26 PM
To: General OpenNMS Discussion <[hidden email]>
Subject: Re: [opennms-discuss] XML collection not working

 

I can understand the confusion. Services need to be discovered (either manually or automatically) first before you can either poll them or collect data from them.

 

What documentation or tutorials have you read through so far? I’d like to know what was missed so that this can be improved for the next person who tries to do this.

 

Ron

 

On Jul 18, 2017, at 10:09 PM, JohnD Blackburn <[hidden email]> wrote:

 

Thankyou for your response.

 

I installed the VTD required packages before I tried setting it as a handler.

 

As suggested, I’ve  removed the handler class entry from collectd-configuration.xml, but it was still not working.

 

I have just now added the following to the default-foreign-source.xml

 

      <detector name="ABBCS-CUPS" class="org.opennms.netmgt.provision.detector.simple.HttpDetector">

         <parameter key="url" value="/printers.xml"/>

         <parameter key="port" value="8000"/>

      </detector>

 

Which seems to have fixed the issue, but then I found I also needed to add a “~” to the “response-text” parameter in poller-configuration.xml:

 

         <parameter key="response-text" value="~.*printer.*"/>

 

 

The documentation for the XML collector doesn’t mention default-foreign-source.xml at all, so is this something that I shouldn’t really need, and I’ve missed something else in my configuration, or is this  just an oversight, and should be in the doco?

 

 

 

 


                                                                                                       

 

 

<image001.gif>

John Blackburn

 

Solution Architect - Cloud Services

 

Enterprise Software

 

 

 

Level 1, 757 Ann Street

 

Fortitude Valley, Queensland, 4006

AUSTRALIA

Phone: +61 7 33033219

 

Mobile: +61 403 344 524

E-Mail:  [hidden email]

 

 

 

From: Ronald Roskens [[hidden email]] 
Sent: Tuesday, 18 July 2017 11:31 PM
To: General OpenNMS Discussion <[hidden email]>
Subject: Re: [opennms-discuss] XML collection not working

 

Does it work if you take out the handler-class parameter in the collectd-configuration.xml? The size of your XML is so small, that I don’t think VTD is going to make that much of a difference in speed for parsing it, so lets try with the default handler first. (You didn’t list which packages you have installed. The VTD handler isn’t installed by default and you need an extra package for it.

 

On Jul 17, 2017, at 12:32 AM, JohnD Blackburn <[hidden email]> wrote:

 

Hi all,

 

I have been trying to implement a custom monitor for CUPS printers using the XML collector, but I’m having problems getting it to collect any data.

 

Can someone please confirm that my configuration is correct, or let me know what I’ve missed.

 

We’re running opennms 20.0.1

 

 

 

I have a webserver set up that returns the metrics I care about in xml format:

 

[root@aue-s-opennms01 etc]# curl http://10.1.2.3:8000/printers.xml

<?xml version="1.0" encoding="UTF-8"?>

<printers>

     <printer name="Printer1">

           <njobs>5</njobs>

     </printer>

     <printer name="Printer2">

           <njobs>0</njobs>

     </printer>

     <printer name="Printer3">

           <njobs>0</njobs>

     </printer>

</printers>

 

 

Here is (I think) all the relevant configuration bits:

 

xml-datacollection-config.xml:

 

    <xml-collection name="ABBCS-CUPS">

        <rrd step="300">

            <rra>RRA:AVERAGE:0.5:1:2016</rra>

            <rra>RRA:AVERAGE:0.5:12:1488</rra>

            <rra>RRA:AVERAGE:0.5:288:366</rra>

            <rra>RRA:MAX:0.5:288:366</rra>

            <rra>RRA:MIN:0.5:288:366</rra>

        </rrd>

        <xml-source url="http://{ipaddr}:8000/printers.xml">

            <import-groups>xml-datacollection/CUPS-Printers.xml</import-groups>

        </xml-source>

    </xml-collection>

 

 

xml-datacollection/CUPS-Printers.xml:

 

<xml-groups>

    <xml-group name="CUPS-Printer-stats" resource-type="CUPSPrinters"

                resource-xpath="/printers/printer"

                key-xpath="@name">

      <xml-object name="printerName" type="string" xpath="@name" />

      <xml-object name="njobs" type="GAUGE" xpath="njobs" />

    </xml-group>

</xml-groups>

 

collectd-configuration.xml:

 

   <package name="ABBCS-CUPS">

      <filter>(IPADDR != '0.0.0.0')</filter>

      <include-range begin="1.1.1.1" end="254.254.254.254"/>

      <include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"/>

      <service name="ABBCS-CUPS" interval="300000" user-defined="false" status="on">

         <parameter key="collection" value="ABBCS-CUPS"/>

         <parameter key="handler-class" value="org.opennms.protocols.xml.vtdxml.DefaultVTDXmlCollectionHandler"/>

      </service>

   </package>

 

 

   <collector service="ABBCS-CUPS" class-name="org.opennms.protocols.xml.collector.XmlCollector"/>

 

 

datacollection-configuration.xml:

 

<?xml version="1.0"?>

<datacollection-config rrdRepository="/opt/opennms/share/rrd/snmp/">

    <snmp-collection name="ABBCS" snmpStorageFlag="select">

        <rrd step="300">

            <rra>RRA:AVERAGE:0.5:1:8928</rra>

            <rra>RRA:AVERAGE:0.5:12:8784</rra>

            <rra>RRA:AVERAGE:0.5:480:366</rra>

            <rra>RRA:MIN:0.5:12:8784</rra>

            <rra>RRA:MAX:0.5:12:8784</rra>

        </rrd>

        <include-collection dataCollectionGroup="ABBCS-CUPS"/>

    </snmp-collection>

</datacollection-config>

 

 

 

datacollection/ABBCS-CUPS.xml:

 

<datacollection-group name="ABBCS-CUPS">

  <resourceType name="CUPSPrinters" label="CUPS Printers"

   resourceLabel="Printer ${printerName}" >

     <persistenceSelectorStrategy

          class="org.opennms.netmgt.collection.support.PersistAllSelectorStrategy"/>

     <storageStrategy

          class="org.opennms.netmgt.collection.support.IndexStorageStrategy"/>

  </resourceType>

</datacollection-group>

 

 

poller-configuration.xml:

 

   <package name="ABBCS-CUPS">

      <filter>((IPADDR != '0.0.0.0') &amp; (catincServers))</filter>

      <include-range begin="1.1.1.1" end="254.254.254.254"/>

      <include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"/>

      <rrd step="300">

         <rra>RRA:AVERAGE:0.5:1:2016</rra>

         <rra>RRA:AVERAGE:0.5:12:1488</rra>

         <rra>RRA:AVERAGE:0.5:288:366</rra>

         <rra>RRA:MAX:0.5:288:366</rra>

         <rra>RRA:MIN:0.5:288:366</rra>

      </rrd>

      <service name="ABBCS-CUPS" interval="300000" user-defined="false" status="on">

         <parameter key="retry" value="1"/>

         <parameter key="timeout" value="3000"/>

         <parameter key="port" value="8000"/>

         <parameter key="url" value="/printers.xml"/>

         <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />

         <parameter key="rrd-base-name" value="cups" />

         <parameter key="response" value="200-202,299"/>

         <parameter key="response-text" value=".*printer.*"/>

      </service>

      <downtime begin="0" end="300000" interval="30000"/>

      <downtime begin="300000" end="43200000" interval="300000"/>

      <downtime begin="43200000" end="432000000" interval="600000"/>

      <downtime begin="432000000" delete="true"/>

   </package>

 

 

   <monitor service="ABBCS-CUPS" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor"/>

 

 

Any assistance greatly appreciated.

 

Regards,

John Blackburn.

 

 

 

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, 
Slashdot.org! http://sdm.link/slashdot_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, 
Slashdot.org! http://sdm.link/slashdot_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: XML collection not working

Ronald Roskens
I’ve added a note under the configuration basics mentioning that you need to add the service to the node either using a provisioning foreign source or as managed service in a provisioning requisition.

I also updated the wiki page to note you don’t need an additional package installed under OpenNMS Horizon 20.

Ron

On Jul 19, 2017, at 8:32 PM, JohnD Blackburn <[hidden email]> wrote:

The main document I used for setting up the XML collection was this one:
 
 
which makes no reference to default foreign source definitions or discovery.
 
I think it needs a section on discovery pointing people to documentation on how to discover the service and/or what methods of discovery would be valid. Also, it is not obvious that XML collection is (potentially) a foreign source so it never occurred to me to search for doco on configuring the default foreign source.
 
Thanks again for your response.
 
Regards,
John

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: XML collection not working

BeeJay

Thanks for that Ronald, excellent work!

 

That bit about not requiring the extra package in V20 had me confused for a little bit when I couldn’t find the package to install J

 

 

John

 

From: Ronald Roskens [mailto:[hidden email]]
Sent: Friday, 21 July 2017 2:05 AM
To: General OpenNMS Discussion <[hidden email]>
Subject: Re: [opennms-discuss] XML collection not working

 

I’ve added a note under the configuration basics mentioning that you need to add the service to the node either using a provisioning foreign source or as managed service in a provisioning requisition.

 

I also updated the wiki page to note you don’t need an additional package installed under OpenNMS Horizon 20.

 

Ron

 

On Jul 19, 2017, at 8:32 PM, JohnD Blackburn <[hidden email]> wrote:

 

The main document I used for setting up the XML collection was this one:

 

 

which makes no reference to default foreign source definitions or discovery.

 

I think it needs a section on discovery pointing people to documentation on how to discover the service and/or what methods of discovery would be valid. Also, it is not obvious that XML collection is (potentially) a foreign source so it never occurred to me to search for doco on configuring the default foreign source.

 

Thanks again for your response.

 

Regards,

John


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss