Re: HTTP Data Collector example doesn't work

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
40 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Data Collector example doesn't work

tezarin
Hi Oka,
 
Thanks for your reply. I replaced all my code with yours but unfortunately it still doesn't collect the data...
 
On the side note, I used mod_status and got the /server-status to work and the http://192.168.1.171/server-status shows all the data. Then followed the instruction on http://www.opennms.org/index.php/Monitoring_Apache_with_the_HTTP_collector and edited all the config files but still no data is being collected....

Please help :)
 
Thanks

 



Date: Wed, 16 Jul 2008 03:05:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I went and updated the Opennms FAQ page with the changes here so this documented.

-Oka

[hidden email] wrote:
I just added the service "with some minor modifications to not disrupt my system" this works and the .jrb file was created.

My capsd-configuration.xml

     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <property key="url" value="/doccount.html" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

collectd-configuration.xml

    <package name="doc-count">
        <filter>IPADDR != '0.0.0.0'</filter>
        <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
            <parameter key="http-collection" value="doc-count-1" />
            <parameter key="retry" value="1" />
            <parameter key="timeout" value="2000" />
            <parameter key="url" value="/doccount.html"/>
         </service>
     </package>

My http-datacollection-config.xml
  <http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/index.html"
             matches="(?s).*?Document\sCount:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

-Oka

Honia A <[hidden email]> wrote:
Hi Oks,
 
Thanks for your reply. I don't have mod_status running and not sure how to get that to work...I have to do a little research on that first. But for now, as I mentioned before, I am also working on the HttpDocCount example and as you may already know, the regex for this example is pretty simple and the data source is almost one line but the data is not being collected... Can you please do me a favor and take a look at the config files below and see what the problem is?
 
doccount.html (located in /var/www/html direcoty):
 

   <html>
     <head>
       <meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
       <meta name="GENERATOR" content="Microsoft FrontPage 4.0">
       <meta name="ProgId" content="FrontPage.Editor.Document">
       <title>collector test</title>
     </head>
     <body>
       <p>Document Count: 5</p>
     </body>
   </html>

 
capsd-configuration.xml (just the part I added):
 
     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

 
collectd-configuration.xml (just the part I added):
 
  <package name="doc-count">
                 <filter>IPADDR != '0.0.0.0'</filter>
                 <include-range begin="1.1.1.1" end="254.254.254.254"/>

                 <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
                        <parameter key="http-collection" value="doc-count-1" />
                        <parameter key="retry" value="1" />
                        <parameter key="timeout" value="2000" />
                 </service>
        </package>

         <collector service="HttpDocCount" class-name="org.opennms.netmgt.collectd.HttpCollector" />
 
 
http-datacollection-config.xml (the entire file)
 
<http-datacollection-config
    xmlns:http-dc="
http://xmlns.opennms.org/xsd/config/http-datacollection"
    xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
http://xmlns.opennms.org/xsd/config/http-datacollection http://www.opennms.org/xsd/config/http-datacollection-config.xsd"
    rrdRepository="/opt/opennms/share/rrd/snmp/" >
  <http-collection name="weather">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:2016</rra>
      <rra>RRA:AVERAGE:0.5:12:1488</rra>
      <rra>RRA:AVERAGE:0.5:288:366</rra>
      <rra>RRA:MAX:0.5:288:366</rra>
      <rra>RRA:MIN:0.5:288:366</rra>
    </rrd>
    <uris>
      <uri name="weather-station">
        <url path="/httpcolltest.html"
             matches="(?s).*?Temperature.*?3366FF">([0-9\.]+).*?Humidity.*?3366FF">([0-9]+)
                     .*?Barometer.*?3366FF">([0-9\.]+).*?Wind\sChill.*?3366FF"><small>([0-9\.]+)
                     .*?Heat\sIndex.*?3366FF"></small>([0-9\.]+).*"
             response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="wsTemperature" match-group="1" type="gauge32"/>
          <attrib alias="wsHumidity"    match-group="2" type="gauge32"/>
          <attrib alias="wsBarometer"   match-group="3" type="gauge32"/>
          <attrib alias="wsWindChill"   match-group="4" type="gauge32"/>
          <attrib alias="wsHeatIndex"   match-group="5" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

<http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/doccount.html"
             matches="(?s).*Document Count:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>
</http-datacollection-config>

 
poller-configuration.xml (just the part I added):
 
    <service name="HttpDocCount" interval="300000" user-defined="true" status="on">
      <parameter key="retry" value="3" />
      <parameter key="timeout" value="10000" />
      <parameter key="port" value="80" />
      <parameter key="url" value="/" />
      <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
      <parameter key="rrd-base-name" value="httpdoccount" />
      <parameter key="ds-name" value="httpdoccount" />
    </service>

 
 
 <monitor service="HttpDocCount" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor" />
 
 
The HttpDocCount service has been discovered on that node (192.168.1.171) and is being monitored properly. The httpdoccount.jrb has been showed up inside /opt/opennms/share/rrd/response/192.168.1.171 direcory but nothing has ever appeared in /opt/opennms/share/rrd/snmp/192.168.1.171 directory which means no data has been ever collected on this node.
 
The project deadline is approaching and I still could't get any of the examples to work. I am sure since you have already worked on more explicit projects, this should be piece of cake for you. Would that be possible for you to help me figuring this out?
 
Thanks in advance
 



Date: Wed, 16 Jul 2008 01:56:36 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I have just updated the page (as in 10 minutes ago) that Pete linked, it contains copy and paste code.

The reason the "page" you created does not work is that mod_status generates a dynamic page so if you have it running /server-status will work under apache.

Starting with the server-status page would be the easiest place to begin as the regex is very simplistic, the weather page is a complex pattern match and you would have to dig through log files to see what its really saying.

-Oka
 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Need to know now? Get instant answers with Windows Live Messenger. IM on your terms.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Use video conversation to talk face-to-face with Windows Live Messenger. Get started.
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Data Collector example doesn't work

okagau007
You will need to paste parts of your collectd.log file search for entries that have this

HttpCollector:

If there is an obvious error you will see a java stack trace associated with it.  Also are you running on the same machine or through a reverse proxy?

This is an example of an error that was generated when there was a misconfigured vhost

XXXX-XX-XX 14:59:52,315 WARN  [CollectdScheduler-50 Pool-fiber1] HttpCollector: doCollection: no attributes defined for collection were found in response t
ext matching regular expression '(?s).*?BusyWorkers:\s([0-9]+).*'
XXXX-XX-XX 14:59:52,315 ERROR [CollectdScheduler-50 Pool-fiber1] HttpCollector: collect: http collection failed: org.opennms.netmgt.collectd.HttpCollector$
HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
org.opennms.netmgt.collectd.HttpCollector$HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
        at org.opennms.netmgt.collectd.HttpCollector.persistResponse(HttpCollector.java:400)
        at org.opennms.netmgt.collectd.HttpCollector.doCollection(HttpCollector.java:253)
        at org.opennms.netmgt.collectd.HttpCollector.access$000(HttpCollector.java:95)
        at org.opennms.netmgt.collectd.HttpCollector$HttpCollectionSet.collect(HttpCollector.java:171)
        at org.opennms.netmgt.collectd.HttpCollector.collect(HttpCollector.java:130)
        at org.opennms.netmgt.collectd.CollectionSpecification.collect(CollectionSpecification.java:223)
        at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:347)
        at org.opennms.netmgt.collectd.CollectableService.run(CollectableService.java:296)
        at org.opennms.netmgt.scheduler.LegacyScheduler$1.run(LegacyScheduler.java:292)

-Oka


Honia A <[hidden email]> wrote:
Hi Oka,
 
Thanks for your reply. I replaced all my code with yours but unfortunately it still doesn't collect the data...
 
On the side note, I used mod_status and got the /server-status to work and the http://192.168.1.171/server-status shows all the data. Then followed the instruction on http://www.opennms.org/index.php/Monitoring_Apache_with_the_HTTP_collector and edited all the config files but still no data is being collected....

Please help :)
 
Thanks

 



Date: Wed, 16 Jul 2008 03:05:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I went and updated the Opennms FAQ page with the changes here so this documented.

-Oka

[hidden email] wrote:
I just added the service "with some minor modifications to not disrupt my system" this works and the .jrb file was created.

My capsd-configuration.xml

     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <property key="url" value="/doccount.html" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

collectd-configuration.xml

    <package name="doc-count">
        <filter>IPADDR != '0.0.0.0'</filter>
        <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
            <parameter key="http-collection" value="doc-count-1" />
            <parameter key="retry" value="1" />
            <parameter key="timeout" value="2000" />
            <parameter key="url" value="/doccount.html"/>
         </service>
     </package>

My http-datacollection-config.xml
  <http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/index.html"
             matches="(?s).*?Document\sCount:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

-Oka

Honia A <[hidden email]> wrote:
Hi Oks,
 
Thanks for your reply. I don't have mod_status running and not sure how to get that to work...I have to do a little research on that first. But for now, as I mentioned before, I am also working on the HttpDocCount example and as you may already know, the regex for this example is pretty simple and the data source is almost one line but the data is not being collected... Can you please do me a favor and take a look at the config files below and see what the problem is?
 
doccount.html (located in /var/www/html direcoty):
 

   <html>
     <head>
       <meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
       <meta name="GENERATOR" content="Microsoft FrontPage 4.0">
       <meta name="ProgId" content="FrontPage.Editor.Document">
       <title>collector test</title>
     </head>
     <body>
       <p>Document Count: 5</p>
     </body>
   </html>

 
capsd-configuration.xml (just the part I added):
 
     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

 
collectd-configuration.xml (just the part I added):
 
  <package name="doc-count">
                 <filter>IPADDR != '0.0.0.0'</filter>
                 <include-range begin="1.1.1.1" end="254.254.254.254"/>

                 <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
                        <parameter key="http-collection" value="doc-count-1" />
                        <parameter key="retry" value="1" />
                        <parameter key="timeout" value="2000" />
                 </service>
        </package>

         <collector service="HttpDocCount" class-name="org.opennms.netmgt.collectd.HttpCollector" />
 
 
http-datacollection-config.xml (the entire file)
 
<http-datacollection-config
    xmlns:http-dc="
http://xmlns.opennms.org/xsd/config/http-datacollection"
    xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
http://xmlns.opennms.org/xsd/config/http-datacollection http://www.opennms.org/xsd/config/http-datacollection-config.xsd"
    rrdRepository="/opt/opennms/share/rrd/snmp/" >
  <http-collection name="weather">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:2016</rra>
      <rra>RRA:AVERAGE:0.5:12:1488</rra>
      <rra>RRA:AVERAGE:0.5:288:366</rra>
      <rra>RRA:MAX:0.5:288:366</rra>
      <rra>RRA:MIN:0.5:288:366</rra>
    </rrd>
    <uris>
      <uri name="weather-station">
        <url path="/httpcolltest.html"
             matches="(?s).*?Temperature.*?3366FF">([0-9\.]+).*?Humidity.*?3366FF">([0-9]+)
                     .*?Barometer.*?3366FF">([0-9\.]+).*?Wind\sChill.*?3366FF"><small>([0-9\.]+)
                     .*?Heat\sIndex.*?3366FF"></small>([0-9\.]+).*"
             response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="wsTemperature" match-group="1" type="gauge32"/>
          <attrib alias="wsHumidity"    match-group="2" type="gauge32"/>
          <attrib alias="wsBarometer"   match-group="3" type="gauge32"/>
          <attrib alias="wsWindChill"   match-group="4" type="gauge32"/>
          <attrib alias="wsHeatIndex"   match-group="5" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

<http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/doccount.html"
             matches="(?s).*Document Count:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>
</http-datacollection-config>

 
poller-configuration.xml (just the part I added):
 
    <service name="HttpDocCount" interval="300000" user-defined="true" status="on">
      <parameter key="retry" value="3" />
      <parameter key="timeout" value="10000" />
      <parameter key="port" value="80" />
      <parameter key="url" value="/" />
      <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
      <parameter key="rrd-base-name" value="httpdoccount" />
      <parameter key="ds-name" value="httpdoccount" />
    </service>

 
 
 <monitor service="HttpDocCount" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor" />
 
 
The HttpDocCount service has been discovered on that node (192.168.1.171) and is being monitored properly. The httpdoccount.jrb has been showed up inside /opt/opennms/share/rrd/response/192.168.1.171 direcory but nothing has ever appeared in /opt/opennms/share/rrd/snmp/192.168.1.171 directory which means no data has been ever collected on this node.
 
The project deadline is approaching and I still could't get any of the examples to work. I am sure since you have already worked on more explicit projects, this should be piece of cake for you. Would that be possible for you to help me figuring this out?
 
Thanks in advance
 



Date: Wed, 16 Jul 2008 01:56:36 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I have just updated the page (as in 10 minutes ago) that Pete linked, it contains copy and paste code.

The reason the "page" you created does not work is that mod_status generates a dynamic page so if you have it running /server-status will work under apache.

Starting with the server-status page would be the easiest place to begin as the regex is very simplistic, the weather page is a complex pattern match and you would have to dig through log files to see what its really saying.

-Oka
 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Need to know now? Get instant answers with Windows Live Messenger. IM on your terms.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Use video conversation to talk face-to-face with Windows Live Messenger. Get started.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 


Stop! Global Warming ~ Yahoo! JAPAN Earth Project

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Data Collector example doesn't work

tezarin
Oka,
 
I looked at the entire log file (collectd.log) but the word "HttpCollector" hasn't been mentioned anywhere.... No Java block either.
 
I use ssh client (Putty) software to connect to the that server which is located on another machine.
 
Please advise.
 
Thanks,
honia
 


 
 



Date: Wed, 16 Jul 2008 05:12:02 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

You will need to paste parts of your collectd.log file search for entries that have this

HttpCollector:

If there is an obvious error you will see a java stack trace associated with it.  Also are you running on the same machine or through a reverse proxy?

This is an example of an error that was generated when there was a misconfigured vhost

XXXX-XX-XX 14:59:52,315 WARN  [CollectdScheduler-50 Pool-fiber1] HttpCollector: doCollection: no attributes defined for collection were found in response t
ext matching regular expression '(?s).*?BusyWorkers:\s([0-9]+).*'
XXXX-XX-XX 14:59:52,315 ERROR [CollectdScheduler-50 Pool-fiber1] HttpCollector: collect: http collection failed: org.opennms.netmgt.collectd.HttpCollector$
HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
org.opennms.netmgt.collectd.HttpCollector$HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
        at org.opennms.netmgt.collectd.HttpCollector.persistResponse(HttpCollector.java:400)
        at org.opennms.netmgt.collectd.HttpCollector.doCollection(HttpCollector.java:253)
        at org.opennms.netmgt.collectd.HttpCollector.access$000(HttpCollector.java:95)
        at org.opennms.netmgt.collectd.HttpCollector$HttpCollectionSet.collect(HttpCollector.java:171)
        at org.opennms.netmgt.collectd.HttpCollector.collect(HttpCollector.java:130)
        at org.opennms.netmgt.collectd.CollectionSpecification.collect(CollectionSpecification.java:223)
        at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:347)
        at org.opennms.netmgt.collectd.CollectableService.run(CollectableService.java:296)
        at org.opennms.netmgt.scheduler.LegacyScheduler$1.run(LegacyScheduler.java:292)

-Oka


Honia A <[hidden email]> wrote:
Hi Oka,
 
Thanks for your reply. I replaced all my code with yours but unfortunately it still doesn't collect the data...
 
On the side note, I used mod_status and got the /server-status to work and the http://192.168.1.171/server-status shows all the data. Then followed the instruction on http://www.opennms.org/index.php/Monitoring_Apache_with_the_HTTP_collector and edited all the config files but still no data is being collected....

Please help :)
 
Thanks

 



Date: Wed, 16 Jul 2008 03:05:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I went and updated the Opennms FAQ page with the changes here so this documented.

-Oka

[hidden email] wrote:
I just added the service "with some minor modifications to not disrupt my system" this works and the .jrb file was created.

My capsd-configuration.xml

     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <property key="url" value="/doccount.html" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

collectd-configuration.xml

    <package name="doc-count">
        <filter>IPADDR != '0.0.0.0'</filter>
        <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
            <parameter key="http-collection" value="doc-count-1" />
            <parameter key="retry" value="1" />
            <parameter key="timeout" value="2000" />
            <parameter key="url" value="/doccount.html"/>
         </service>
     </package>

My http-datacollection-config.xml
  <http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/index.html"
             matches="(?s).*?Document\sCount:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

-Oka

Honia A <[hidden email]> wrote:
Hi Oks,
 
Thanks for your reply. I don't have mod_status running and not sure how to get that to work...I have to do a little research on that first. But for now, as I mentioned before, I am also working on the HttpDocCount example and as you may already know, the regex for this example is pretty simple and the data source is almost one line but the data is not being collected... Can you please do me a favor and take a look at the config files below and see what the problem is?
 
doccount.html (located in /var/www/html direcoty):
 

   <html>
     <head>
       <meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
       <meta name="GENERATOR" content="Microsoft FrontPage 4.0">
       <meta name="ProgId" content="FrontPage.Editor.Document">
       <title>collector test</title>
     </head>
     <body>
       <p>Document Count: 5</p>
     </body>
   </html>

 
capsd-configuration.xml (just the part I added):
 
     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

 
collectd-configuration.xml (just the part I added):
 
  <package name="doc-count">
                 <filter>IPADDR != '0.0.0.0'</filter>
                 <include-range begin="1.1.1.1" end="254.254.254.254"/>

                 <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
                        <parameter key="http-collection" value="doc-count-1" />
                        <parameter key="retry" value="1" />
                        <parameter key="timeout" value="2000" />
                 </service>
        </package>

         <collector service="HttpDocCount" class-name="org.opennms.netmgt.collectd.HttpCollector" />
 
 
http-datacollection-config.xml (the entire file)
 
<http-datacollection-config
    xmlns:http-dc="
http://xmlns.opennms.org/xsd/config/http-datacollection"
    xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
http://xmlns.opennms.org/xsd/config/http-datacollection http://www.opennms.org/xsd/config/http-datacollection-config.xsd"
    rrdRepository="/opt/opennms/share/rrd/snmp/" >
  <http-collection name="weather">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:2016</rra>
      <rra>RRA:AVERAGE:0.5:12:1488</rra>
      <rra>RRA:AVERAGE:0.5:288:366</rra>
      <rra>RRA:MAX:0.5:288:366</rra>
      <rra>RRA:MIN:0.5:288:366</rra>
    </rrd>
    <uris>
      <uri name="weather-station">
        <url path="/httpcolltest.html"
             matches="(?s).*?Temperature.*?3366FF">([0-9\.]+).*?Humidity.*?3366FF">([0-9]+)
                     .*?Barometer.*?3366FF">([0-9\.]+).*?Wind\sChill.*?3366FF"><small>([0-9\.]+)
                     .*?Heat\sIndex.*?3366FF"></small>([0-9\.]+).*"
             response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="wsTemperature" match-group="1" type="gauge32"/>
          <attrib alias="wsHumidity"    match-group="2" type="gauge32"/>
          <attrib alias="wsBarometer"   match-group="3" type="gauge32"/>
          <attrib alias="wsWindChill"   match-group="4" type="gauge32"/>
          <attrib alias="wsHeatIndex"   match-group="5" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

<http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/doccount.html"
             matches="(?s).*Document Count:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>
</http-datacollection-config>

 
poller-configuration.xml (just the part I added):
 
    <service name="HttpDocCount" interval="300000" user-defined="true" status="on">
      <parameter key="retry" value="3" />
      <parameter key="timeout" value="10000" />
      <parameter key="port" value="80" />
      <parameter key="url" value="/" />
      <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
      <parameter key="rrd-base-name" value="httpdoccount" />
      <parameter key="ds-name" value="httpdoccount" />
    </service>

 
 
 <monitor service="HttpDocCount" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor" />
 
 
The HttpDocCount service has been discovered on that node (192.168.1.171) and is being monitored properly. The httpdoccount.jrb has been showed up inside /opt/opennms/share/rrd/response/192.168.1.171 direcory but nothing has ever appeared in /opt/opennms/share/rrd/snmp/192.168.1.171 directory which means no data has been ever collected on this node.
 
The project deadline is approaching and I still could't get any of the examples to work. I am sure since you have already worked on more explicit projects, this should be piece of cake for you. Would that be possible for you to help me figuring this out?
 
Thanks in advance
 



Date: Wed, 16 Jul 2008 01:56:36 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I have just updated the page (as in 10 minutes ago) that Pete linked, it contains copy and paste code.

The reason the "page" you created does not work is that mod_status generates a dynamic page so if you have it running /server-status will work under apache.

Starting with the server-status page would be the easiest place to begin as the regex is very simplistic, the weather page is a complex pattern match and you would have to dig through log files to see what its really saying.

-Oka
 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Need to know now? Get instant answers with Windows Live Messenger. IM on your terms.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Use video conversation to talk face-to-face with Windows Live Messenger. Get started.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


With Windows Live for mobile, your contacts travel with you. Connect on the go.
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Data Collector example doesn't work

okagau007
Honia,

Go into the etc directory and locate the file log4j.properties, and turn on DEBUG for the collectd.log. You shouldn't need to restart NMS for this, force a rescan on the node and then examine the collectd log.

log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log



Honia A <[hidden email]> wrote:
Oka,
 
I looked at the entire log file (collectd.log) but the word "HttpCollector" hasn't been mentioned anywhere.... No Java block either.
 
I use ssh client (Putty) software to connect to the that server which is located on another machine.
 
Please advise.
 
Thanks,
honia
 


 
 



Date: Wed, 16 Jul 2008 05:12:02 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

You will need to paste parts of your collectd.log file search for entries that have this

HttpCollector:

If there is an obvious error you will see a java stack trace associated with it.  Also are you running on the same machine or through a reverse proxy?

This is an example of an error that was generated when there was a misconfigured vhost

XXXX-XX-XX 14:59:52,315 WARN  [CollectdScheduler-50 Pool-fiber1] HttpCollector: doCollection: no attributes defined for collection were found in response t
ext matching regular expression '(?s).*?BusyWorkers:\s([0-9]+).*'
XXXX-XX-XX 14:59:52,315 ERROR [CollectdScheduler-50 Pool-fiber1] HttpCollector: collect: http collection failed: org.opennms.netmgt.collectd.HttpCollector$
HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
org.opennms.netmgt.collectd.HttpCollector$HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
        at org.opennms.netmgt.collectd.HttpCollector.persistResponse(HttpCollector.java:400)
        at org.opennms.netmgt.collectd.HttpCollector.doCollection(HttpCollector.java:253)
        at org.opennms.netmgt.collectd.HttpCollector.access$000(HttpCollector.java:95)
        at org.opennms.netmgt.collectd.HttpCollector$HttpCollectionSet.collect(HttpCollector.java:171)
        at org.opennms.netmgt.collectd.HttpCollector.collect(HttpCollector.java:130)
        at org.opennms.netmgt.collectd.CollectionSpecification.collect(CollectionSpecification.java:223)
        at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:347)
        at org.opennms.netmgt.collectd.CollectableService.run(CollectableService.java:296)
        at org.opennms.netmgt.scheduler.LegacyScheduler$1.run(LegacyScheduler.java:292)

-Oka


Honia A <[hidden email]> wrote:
Hi Oka,
 
Thanks for your reply. I replaced all my code with yours but unfortunately it still doesn't collect the data...
 
On the side note, I used mod_status and got the /server-status to work and the http://192.168.1.171/server-status shows all the data. Then followed the instruction on http://www.opennms.org/index.php/Monitoring_Apache_with_the_HTTP_collector and edited all the config files but still no data is being collected....

Please help :)
 
Thanks

 



Date: Wed, 16 Jul 2008 03:05:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I went and updated the Opennms FAQ page with the changes here so this documented.

-Oka

[hidden email] wrote:
I just added the service "with some minor modifications to not disrupt my system" this works and the .jrb file was created.

My capsd-configuration.xml

     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <property key="url" value="/doccount.html" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

collectd-configuration.xml

    <package name="doc-count">
        <filter>IPADDR != '0.0.0.0'</filter>
        <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
            <parameter key="http-collection" value="doc-count-1" />
            <parameter key="retry" value="1" />
            <parameter key="timeout" value="2000" />
            <parameter key="url" value="/doccount.html"/>
         </service>
     </package>

My http-datacollection-config.xml
  <http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/index.html"
             matches="(?s).*?Document\sCount:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

-Oka

Honia A <[hidden email]> wrote:
Hi Oks,
 
Thanks for your reply. I don't have mod_status running and not sure how to get that to work...I have to do a little research on that first. But for now, as I mentioned before, I am also working on the HttpDocCount example and as you may already know, the regex for this example is pretty simple and the data source is almost one line but the data is not being collected... Can you please do me a favor and take a look at the config files below and see what the problem is?
 
doccount.html (located in /var/www/html direcoty):
 

   <html>
     <head>
       <meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
       <meta name="GENERATOR" content="Microsoft FrontPage 4.0">
       <meta name="ProgId" content="FrontPage.Editor.Document">
       <title>collector test</title>
     </head>
     <body>
       <p>Document Count: 5</p>
     </body>
   </html>

 
capsd-configuration.xml (just the part I added):
 
     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

 
collectd-configuration.xml (just the part I added):
 
  <package name="doc-count">
                 <filter>IPADDR != '0.0.0.0'</filter>
                 <include-range begin="1.1.1.1" end="254.254.254.254"/>

                 <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
                        <parameter key="http-collection" value="doc-count-1" />
                        <parameter key="retry" value="1" />
                        <parameter key="timeout" value="2000" />
                 </service>
        </package>

         <collector service="HttpDocCount" class-name="org.opennms.netmgt.collectd.HttpCollector" />
 
 
http-datacollection-config.xml (the entire file)
 
<http-datacollection-config
    xmlns:http-dc="
http://xmlns.opennms.org/xsd/config/http-datacollection"
    xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
http://xmlns.opennms.org/xsd/config/http-datacollection http://www.opennms.org/xsd/config/http-datacollection-config.xsd"
    rrdRepository="/opt/opennms/share/rrd/snmp/" >
  <http-collection name="weather">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:2016</rra>
      <rra>RRA:AVERAGE:0.5:12:1488</rra>
      <rra>RRA:AVERAGE:0.5:288:366</rra>
      <rra>RRA:MAX:0.5:288:366</rra>
      <rra>RRA:MIN:0.5:288:366</rra>
    </rrd>
    <uris>
      <uri name="weather-station">
        <url path="/httpcolltest.html"
             matches="(?s).*?Temperature.*?3366FF">([0-9\.]+).*?Humidity.*?3366FF">([0-9]+)
                     .*?Barometer.*?3366FF">([0-9\.]+).*?Wind\sChill.*?3366FF"><small>([0-9\.]+)
                     .*?Heat\sIndex.*?3366FF"></small>([0-9\.]+).*"
             response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="wsTemperature" match-group="1" type="gauge32"/>
          <attrib alias="wsHumidity"    match-group="2" type="gauge32"/>
          <attrib alias="wsBarometer"   match-group="3" type="gauge32"/>
          <attrib alias="wsWindChill"   match-group="4" type="gauge32"/>
          <attrib alias="wsHeatIndex"   match-group="5" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

<http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/doccount.html"
             matches="(?s).*Document Count:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>
</http-datacollection-config>

 
poller-configuration.xml (just the part I added):
 
    <service name="HttpDocCount" interval="300000" user-defined="true" status="on">
      <parameter key="retry" value="3" />
      <parameter key="timeout" value="10000" />
      <parameter key="port" value="80" />
      <parameter key="url" value="/" />
      <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
      <parameter key="rrd-base-name" value="httpdoccount" />
      <parameter key="ds-name" value="httpdoccount" />
    </service>

 
 
 <monitor service="HttpDocCount" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor" />
 
 
The HttpDocCount service has been discovered on that node (192.168.1.171) and is being monitored properly. The httpdoccount.jrb has been showed up inside /opt/opennms/share/rrd/response/192.168.1.171 direcory but nothing has ever appeared in /opt/opennms/share/rrd/snmp/192.168.1.171 directory which means no data has been ever collected on this node.
 
The project deadline is approaching and I still could't get any of the examples to work. I am sure since you have already worked on more explicit projects, this should be piece of cake for you. Would that be possible for you to help me figuring this out?
 
Thanks in advance
 



Date: Wed, 16 Jul 2008 01:56:36 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I have just updated the page (as in 10 minutes ago) that Pete linked, it contains copy and paste code.

The reason the "page" you created does not work is that mod_status generates a dynamic page so if you have it running /server-status will work under apache.

Starting with the server-status page would be the easiest place to begin as the regex is very simplistic, the weather page is a complex pattern match and you would have to dig through log files to see what its really saying.

-Oka
 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Need to know now? Get instant answers with Windows Live Messenger. IM on your terms.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Use video conversation to talk face-to-face with Windows Live Messenger. Get started.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


With Windows Live for mobile, your contacts travel with you. Connect on the go.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 


Stop! Global Warming ~ Yahoo! JAPAN Earth Project

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Data Collector example doesn't work

tezarin
Oka,
 
This block of code already exist in my log4j.properties and looks like it's been turned on:
 
# Collectd
log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log
log4j.appender.COLLECTD.layout=org.apache.log4j.PatternLayout
log4j.appender.COLLECTD.layout.ConversionPattern=%d %-5p [%t] %c{1}: %m%n

 
 
About the rescan, I had changed the default value for rescan-frequency in capsd-configuration....it's now 10. I did that so that opennms discover the service immediately...
 
Please advise.
 
Thanks



 



Date: Wed, 16 Jul 2008 22:07:56 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Go into the etc directory and locate the file log4j.properties, and turn on DEBUG for the collectd.log. You shouldn't need to restart NMS for this, force a rescan on the node and then examine the collectd log.

log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log



Honia A <[hidden email]> wrote:
Oka,
 
I looked at the entire log file (collectd.log) but the word "HttpCollector" hasn't been mentioned anywhere.... No Java block either.
 
I use ssh client (Putty) software to connect to the that server which is located on another machine.
 
Please advise.
 
Thanks,
honia
 


 
 



Date: Wed, 16 Jul 2008 05:12:02 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

You will need to paste parts of your collectd.log file search for entries that have this

HttpCollector:

If there is an obvious error you will see a java stack trace associated with it.  Also are you running on the same machine or through a reverse proxy?

This is an example of an error that was generated when there was a misconfigured vhost

XXXX-XX-XX 14:59:52,315 WARN  [CollectdScheduler-50 Pool-fiber1] HttpCollector: doCollection: no attributes defined for collection were found in response t
ext matching regular expression '(?s).*?BusyWorkers:\s([0-9]+).*'
XXXX-XX-XX 14:59:52,315 ERROR [CollectdScheduler-50 Pool-fiber1] HttpCollector: collect: http collection failed: org.opennms.netmgt.collectd.HttpCollector$
HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
org.opennms.netmgt.collectd.HttpCollector$HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
        at org.opennms.netmgt.collectd.HttpCollector.persistResponse(HttpCollector.java:400)
        at org.opennms.netmgt.collectd.HttpCollector.doCollection(HttpCollector.java:253)
        at org.opennms.netmgt.collectd.HttpCollector.access$000(HttpCollector.java:95)
        at org.opennms.netmgt.collectd.HttpCollector$HttpCollectionSet.collect(HttpCollector.java:171)
        at org.opennms.netmgt.collectd.HttpCollector.collect(HttpCollector.java:130)
        at org.opennms.netmgt.collectd.CollectionSpecification.collect(CollectionSpecification.java:223)
        at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:347)
        at org.opennms.netmgt.collectd.CollectableService.run(CollectableService.java:296)
        at org.opennms.netmgt.scheduler.LegacyScheduler$1.run(LegacyScheduler.java:292)

-Oka


Honia A <[hidden email]> wrote:
Hi Oka,
 
Thanks for your reply. I replaced all my code with yours but unfortunately it still doesn't collect the data...
 
On the side note, I used mod_status and got the /server-status to work and the http://192.168.1.171/server-status shows all the data. Then followed the instruction on http://www.opennms.org/index.php/Monitoring_Apache_with_the_HTTP_collector and edited all the config files but still no data is being collected....

Please help :)
 
Thanks

 



Date: Wed, 16 Jul 2008 03:05:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I went and updated the Opennms FAQ page with the changes here so this documented.

-Oka

[hidden email] wrote:
I just added the service "with some minor modifications to not disrupt my system" this works and the .jrb file was created.

My capsd-configuration.xml

     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <property key="url" value="/doccount.html" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

collectd-configuration.xml

    <package name="doc-count">
        <filter>IPADDR != '0.0.0.0'</filter>
        <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
            <parameter key="http-collection" value="doc-count-1" />
            <parameter key="retry" value="1" />
            <parameter key="timeout" value="2000" />
            <parameter key="url" value="/doccount.html"/>
         </service>
     </package>

My http-datacollection-config.xml
  <http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/index.html"
             matches="(?s).*?Document\sCount:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

-Oka

Honia A <[hidden email]> wrote:
Hi Oks,
 
Thanks for your reply. I don't have mod_status running and not sure how to get that to work...I have to do a little research on that first. But for now, as I mentioned before, I am also working on the HttpDocCount example and as you may already know, the regex for this example is pretty simple and the data source is almost one line but the data is not being collected... Can you please do me a favor and take a look at the config files below and see what the problem is?
 
doccount.html (located in /var/www/html direcoty):
 

   <html>
     <head>
       <meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
       <meta name="GENERATOR" content="Microsoft FrontPage 4.0">
       <meta name="ProgId" content="FrontPage.Editor.Document">
       <title>collector test</title>
     </head>
     <body>
       <p>Document Count: 5</p>
     </body>
   </html>

 
capsd-configuration.xml (just the part I added):
 
     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

 
collectd-configuration.xml (just the part I added):
 
  <package name="doc-count">
                 <filter>IPADDR != '0.0.0.0'</filter>
                 <include-range begin="1.1.1.1" end="254.254.254.254"/>

                 <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
                        <parameter key="http-collection" value="doc-count-1" />
                        <parameter key="retry" value="1" />
                        <parameter key="timeout" value="2000" />
                 </service>
        </package>

         <collector service="HttpDocCount" class-name="org.opennms.netmgt.collectd.HttpCollector" />
 
 
http-datacollection-config.xml (the entire file)
 
<http-datacollection-config
    xmlns:http-dc="
http://xmlns.opennms.org/xsd/config/http-datacollection"
    xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
http://xmlns.opennms.org/xsd/config/http-datacollection http://www.opennms.org/xsd/config/http-datacollection-config.xsd"
    rrdRepository="/opt/opennms/share/rrd/snmp/" >
  <http-collection name="weather">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:2016</rra>
      <rra>RRA:AVERAGE:0.5:12:1488</rra>
      <rra>RRA:AVERAGE:0.5:288:366</rra>
      <rra>RRA:MAX:0.5:288:366</rra>
      <rra>RRA:MIN:0.5:288:366</rra>
    </rrd>
    <uris>
      <uri name="weather-station">
        <url path="/httpcolltest.html"
             matches="(?s).*?Temperature.*?3366FF">([0-9\.]+).*?Humidity.*?3366FF">([0-9]+)
                     .*?Barometer.*?3366FF">([0-9\.]+).*?Wind\sChill.*?3366FF"><small>([0-9\.]+)
                     .*?Heat\sIndex.*?3366FF"></small>([0-9\.]+).*"
             response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="wsTemperature" match-group="1" type="gauge32"/>
          <attrib alias="wsHumidity"    match-group="2" type="gauge32"/>
          <attrib alias="wsBarometer"   match-group="3" type="gauge32"/>
          <attrib alias="wsWindChill"   match-group="4" type="gauge32"/>
          <attrib alias="wsHeatIndex"   match-group="5" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

<http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/doccount.html"
             matches="(?s).*Document Count:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>
</http-datacollection-config>

 
poller-configuration.xml (just the part I added):
 
    <service name="HttpDocCount" interval="300000" user-defined="true" status="on">
      <parameter key="retry" value="3" />
      <parameter key="timeout" value="10000" />
      <parameter key="port" value="80" />
      <parameter key="url" value="/" />
      <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
      <parameter key="rrd-base-name" value="httpdoccount" />
      <parameter key="ds-name" value="httpdoccount" />
    </service>

 
 
 <monitor service="HttpDocCount" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor" />
 
 
The HttpDocCount service has been discovered on that node (192.168.1.171) and is being monitored properly. The httpdoccount.jrb has been showed up inside /opt/opennms/share/rrd/response/192.168.1.171 direcory but nothing has ever appeared in /opt/opennms/share/rrd/snmp/192.168.1.171 directory which means no data has been ever collected on this node.
 
The project deadline is approaching and I still could't get any of the examples to work. I am sure since you have already worked on more explicit projects, this should be piece of cake for you. Would that be possible for you to help me figuring this out?
 
Thanks in advance
 



Date: Wed, 16 Jul 2008 01:56:36 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I have just updated the page (as in 10 minutes ago) that Pete linked, it contains copy and paste code.

The reason the "page" you created does not work is that mod_status generates a dynamic page so if you have it running /server-status will work under apache.

Starting with the server-status page would be the easiest place to begin as the regex is very simplistic, the weather page is a complex pattern match and you would have to dig through log files to see what its really saying.

-Oka
 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Need to know now? Get instant answers with Windows Live Messenger. IM on your terms.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Use video conversation to talk face-to-face with Windows Live Messenger. Get started.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


With Windows Live for mobile, your contacts travel with you. Connect on the go.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Time for vacation? WIN what you need. Enter Now!
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Data Collector example doesn't work

Jeff Gehlbach
On Jul 16, 2008, at 9:24 AM, Honia A wrote:
> About the rescan, I had changed the default value for rescan-
> frequency in capsd-configuration....it's now 10. I did that so that  
> opennms discover the service immediately...

You set rescan-frequency to a value of 10?  Like:

<capsd-configuration
         rescan-frequency="10" ... >


?

Don't do that!  You will have Capsd trying to rescan every node every  
ten milliseconds.

-jeff

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Data Collector example doesn't work

tezarin
It was a temporary change, I just wanted the service to be discovered faster... I just changed it back to its default value :) Thanks Jeff:)

 



> From: [hidden email]

> To: [hidden email]
> Date: Wed, 16 Jul 2008 09:55:35 -0400
> Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work
>
> On Jul 16, 2008, at 9:24 AM, Honia A wrote:
> > About the rescan, I had changed the default value for rescan-
> > frequency in capsd-configuration....it's now 10. I did that so that
> > opennms discover the service immediately...
>
> You set rescan-frequency to a value of 10? Like:
>
> <capsd-configuration
> rescan-frequency="10" ... >
>
>
> ?
>
> Don't do that! You will have Capsd trying to rescan every node every
> ten milliseconds.
>
> -jeff
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Please read the OpenNMS Mailing List FAQ:
> http://www.opennms.org/index.php/Mailing_List_FAQ
>
> opennms-discuss mailing list
>
> To *unsubscribe* or change your subscription options, see the bottom of this page:
> https://lists.sourceforge.net/lists/listinfo/opennms-discuss


Stay in touch when you're away with Windows Live Messenger. IM anytime you're online.
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Data Collector example doesn't work

okagau007
In reply to this post by tezarin
Honia,

Which version of NMS are you using?

-Oka

Honia A <[hidden email]> wrote:
Oka,
 
This block of code already exist in my log4j.properties and looks like it's been turned on:
 
# Collectd
log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log
log4j.appender.COLLECTD.layout=org.apache.log4j.PatternLayout
log4j.appender.COLLECTD.layout.ConversionPattern=%d %-5p [%t] %c{1}: %m%n

 
 
About the rescan, I had changed the default value for rescan-frequency in capsd-configuration....it's now 10. I did that so that opennms discover the service immediately...
 
Please advise.
 
Thanks



 



Date: Wed, 16 Jul 2008 22:07:56 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Go into the etc directory and locate the file log4j.properties, and turn on DEBUG for the collectd.log. You shouldn't need to restart NMS for this, force a rescan on the node and then examine the collectd log.

log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log



Honia A <[hidden email]> wrote:
Oka,
 
I looked at the entire log file (collectd.log) but the word "HttpCollector" hasn't been mentioned anywhere.... No Java block either.
 
I use ssh client (Putty) software to connect to the that server which is located on another machine.
 
Please advise.
 
Thanks,
honia
 


 
 



Date: Wed, 16 Jul 2008 05:12:02 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

You will need to paste parts of your collectd.log file search for entries that have this

HttpCollector:

If there is an obvious error you will see a java stack trace associated with it.  Also are you running on the same machine or through a reverse proxy?

This is an example of an error that was generated when there was a misconfigured vhost

XXXX-XX-XX 14:59:52,315 WARN  [CollectdScheduler-50 Pool-fiber1] HttpCollector: doCollection: no attributes defined for collection were found in response t
ext matching regular expression '(?s).*?BusyWorkers:\s([0-9]+).*'
XXXX-XX-XX 14:59:52,315 ERROR [CollectdScheduler-50 Pool-fiber1] HttpCollector: collect: http collection failed: org.opennms.netmgt.collectd.HttpCollector$
HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
org.opennms.netmgt.collectd.HttpCollector$HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
        at org.opennms.netmgt.collectd.HttpCollector.persistResponse(HttpCollector.java:400)
        at org.opennms.netmgt.collectd.HttpCollector.doCollection(HttpCollector.java:253)
        at org.opennms.netmgt.collectd.HttpCollector.access$000(HttpCollector.java:95)
        at org.opennms.netmgt.collectd.HttpCollector$HttpCollectionSet.collect(HttpCollector.java:171)
        at org.opennms.netmgt.collectd.HttpCollector.collect(HttpCollector.java:130)
        at org.opennms.netmgt.collectd.CollectionSpecification.collect(CollectionSpecification.java:223)
        at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:347)
        at org.opennms.netmgt.collectd.CollectableService.run(CollectableService.java:296)
        at org.opennms.netmgt.scheduler.LegacyScheduler$1.run(LegacyScheduler.java:292)

-Oka


Honia A <[hidden email]> wrote:
Hi Oka,
 
Thanks for your reply. I replaced all my code with yours but unfortunately it still doesn't collect the data...
 
On the side note, I used mod_status and got the /server-status to work and the http://192.168.1.171/server-status shows all the data. Then followed the instruction on http://www.opennms.org/index.php/Monitoring_Apache_with_the_HTTP_collector and edited all the config files but still no data is being collected....

Please help :)
 
Thanks

 



Date: Wed, 16 Jul 2008 03:05:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I went and updated the Opennms FAQ page with the changes here so this documented.

-Oka

[hidden email] wrote:
I just added the service "with some minor modifications to not disrupt my system" this works and the .jrb file was created.

My capsd-configuration.xml

     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <property key="url" value="/doccount.html" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

collectd-configuration.xml

    <package name="doc-count">
        <filter>IPADDR != '0.0.0.0'</filter>
        <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
            <parameter key="http-collection" value="doc-count-1" />
            <parameter key="retry" value="1" />
            <parameter key="timeout" value="2000" />
            <parameter key="url" value="/doccount.html"/>
         </service>
     </package>

My http-datacollection-config.xml
  <http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/index.html"
             matches="(?s).*?Document\sCount:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

-Oka

Honia A <[hidden email]> wrote:
Hi Oks,
 
Thanks for your reply. I don't have mod_status running and not sure how to get that to work...I have to do a little research on that first. But for now, as I mentioned before, I am also working on the HttpDocCount example and as you may already know, the regex for this example is pretty simple and the data source is almost one line but the data is not being collected... Can you please do me a favor and take a look at the config files below and see what the problem is?
 
doccount.html (located in /var/www/html direcoty):
 

   <html>
     <head>
       <meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
       <meta name="GENERATOR" content="Microsoft FrontPage 4.0">
       <meta name="ProgId" content="FrontPage.Editor.Document">
       <title>collector test</title>
     </head>
     <body>
       <p>Document Count: 5</p>
     </body>
   </html>

 
capsd-configuration.xml (just the part I added):
 
     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

 
collectd-configuration.xml (just the part I added):
 
  <package name="doc-count">
                 <filter>IPADDR != '0.0.0.0'</filter>
                 <include-range begin="1.1.1.1" end="254.254.254.254"/>

                 <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
                        <parameter key="http-collection" value="doc-count-1" />
                        <parameter key="retry" value="1" />
                        <parameter key="timeout" value="2000" />
                 </service>
        </package>

         <collector service="HttpDocCount" class-name="org.opennms.netmgt.collectd.HttpCollector" />
 
 
http-datacollection-config.xml (the entire file)
 
<http-datacollection-config
    xmlns:http-dc="
http://xmlns.opennms.org/xsd/config/http-datacollection"
    xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
http://xmlns.opennms.org/xsd/config/http-datacollection http://www.opennms.org/xsd/config/http-datacollection-config.xsd"
    rrdRepository="/opt/opennms/share/rrd/snmp/" >
  <http-collection name="weather">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:2016</rra>
      <rra>RRA:AVERAGE:0.5:12:1488</rra>
      <rra>RRA:AVERAGE:0.5:288:366</rra>
      <rra>RRA:MAX:0.5:288:366</rra>
      <rra>RRA:MIN:0.5:288:366</rra>
    </rrd>
    <uris>
      <uri name="weather-station">
        <url path="/httpcolltest.html"
             matches="(?s).*?Temperature.*?3366FF">([0-9\.]+).*?Humidity.*?3366FF">([0-9]+)
                     .*?Barometer.*?3366FF">([0-9\.]+).*?Wind\sChill.*?3366FF"><small>([0-9\.]+)
                     .*?Heat\sIndex.*?3366FF"></small>([0-9\.]+).*"
             response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="wsTemperature" match-group="1" type="gauge32"/>
          <attrib alias="wsHumidity"    match-group="2" type="gauge32"/>
          <attrib alias="wsBarometer"   match-group="3" type="gauge32"/>
          <attrib alias="wsWindChill"   match-group="4" type="gauge32"/>
          <attrib alias="wsHeatIndex"   match-group="5" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

<http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/doccount.html"
             matches="(?s).*Document Count:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>
</http-datacollection-config>

 
poller-configuration.xml (just the part I added):
 
    <service name="HttpDocCount" interval="300000" user-defined="true" status="on">
      <parameter key="retry" value="3" />
      <parameter key="timeout" value="10000" />
      <parameter key="port" value="80" />
      <parameter key="url" value="/" />
      <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
      <parameter key="rrd-base-name" value="httpdoccount" />
      <parameter key="ds-name" value="httpdoccount" />
    </service>

 
 
 <monitor service="HttpDocCount" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor" />
 
 
The HttpDocCount service has been discovered on that node (192.168.1.171) and is being monitored properly. The httpdoccount.jrb has been showed up inside /opt/opennms/share/rrd/response/192.168.1.171 direcory but nothing has ever appeared in /opt/opennms/share/rrd/snmp/192.168.1.171 directory which means no data has been ever collected on this node.
 
The project deadline is approaching and I still could't get any of the examples to work. I am sure since you have already worked on more explicit projects, this should be piece of cake for you. Would that be possible for you to help me figuring this out?
 
Thanks in advance
 



Date: Wed, 16 Jul 2008 01:56:36 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I have just updated the page (as in 10 minutes ago) that Pete linked, it contains copy and paste code.

The reason the "page" you created does not work is that mod_status generates a dynamic page so if you have it running /server-status will work under apache.

Starting with the server-status page would be the easiest place to begin as the regex is very simplistic, the weather page is a complex pattern match and you would have to dig through log files to see what its really saying.

-Oka
 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Need to know now? Get instant answers with Windows Live Messenger. IM on your terms.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Use video conversation to talk face-to-face with Windows Live Messenger. Get started.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


With Windows Live for mobile, your contacts travel with you. Connect on the go.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Time for vacation? WIN what you need. Enter Now!-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 


Stop! Global Warming ~ Yahoo! JAPAN Earth Project

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Data Collector example doesn't work

tezarin
In reply to this post by okagau007
Oka,
 
Update! I was just browsing the "etc" directory and found something a bit odd...there are "two" collectd-configuration.xml files in that directory. One is called collectd-configuration.xml which has all the data I posted here and the other file is the default version which has the example package only! Could this whole situation be cause by this duplicate file thing?!! I just copied everything from the modified file to this empty file and forced the rescan...guess what? collectd-configuration.xml is now showing the HttpDataCollector message you mentioned earlier! Here's the error message I found in collectd.log file (it's a bit too long so I just paste some of it here for your review).
 
It should be really easy to fix now...please help me :)
 
2008-07-16 11:41:50,355 DEBUG [Main] HttpCollector: initialize: Initializing HttpCollector.
2008-07-16 11:41:50,355 DEBUG [Main] HttpCollector: initialize: Initializing collector: class org.opennms.netmgt.collectd.HttpCollector
2008-07-16 11:41:50,358 DEBUG [Main] Collectd: initialize: initializing http collection config factory.
2008-07-16 11:41:50,433 FATAL [Main] HttpCollector: initialize: Error marshalling configuration.
org.exolab.castor.xml.MarshalException: unable to find FieldDescriptor for 'small' in ClassDescriptor of url{File: [not available]; line: 18; column: 87}
 at org.exolab.castor.xml.Unmarshaller.convertSAXExceptionToMarshalException(Unmarshaller.java:755)
 at org.exolab.castor.xml.Unmarshaller.unmarshal(Unmarshaller.java:721)
 at org.exolab.castor.xml.Unmarshaller.unmarshal(Unmarshaller.java:610)
 at org.exolab.castor.xml.Unmarshaller.unmarshal(Unmarshaller.java:812)
 at org.opennms.netmgt.config.HttpCollectionConfigFactory.initialize(HttpCollectionConfigFactory.java:96)
 at org.opennms.netmgt.config.HttpCollectionConfigFactory.<init>(HttpCollectionConfigFactory.java:84)
 at org.opennms.netmgt.config.HttpCollectionConfigFactory.init(HttpCollectionConfigFactory.java:104)
 at org.opennms.netmgt.collectd.HttpCollector.initHttpCollecionConfig(HttpCollector.java:519)
 at org.opennms.netmgt.collectd.HttpCollector.initialize(HttpCollector.java:511)
 at org.opennms.netmgt.collectd.Collectd.instantiateCollectors(Collectd.java:1379)
 at org.opennms.netmgt.collectd.Collectd.onInit(Collectd.java:191)
 at org.opennms.netmgt.daemon.AbstractServiceDaemon.init(AbstractServiceDaemon.java:201)
 at org.opennms.netmgt.daemon.AbstractServiceDaemon.afterPropertiesSet(AbstractServiceDaemon.java:50)
 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods


 
 



Date: Wed, 16 Jul 2008 22:07:56 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Go into the etc directory and locate the file log4j.properties, and turn on DEBUG for the collectd.log. You shouldn't need to restart NMS for this, force a rescan on the node and then examine the collectd log.

log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log



Honia A <[hidden email]> wrote:
Oka,
 
I looked at the entire log file (collectd.log) but the word "HttpCollector" hasn't been mentioned anywhere.... No Java block either.
 
I use ssh client (Putty) software to connect to the that server which is located on another machine.
 
Please advise.
 
Thanks,
honia
 


 
 



Date: Wed, 16 Jul 2008 05:12:02 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

You will need to paste parts of your collectd.log file search for entries that have this

HttpCollector:

If there is an obvious error you will see a java stack trace associated with it.  Also are you running on the same machine or through a reverse proxy?

This is an example of an error that was generated when there was a misconfigured vhost

XXXX-XX-XX 14:59:52,315 WARN  [CollectdScheduler-50 Pool-fiber1] HttpCollector: doCollection: no attributes defined for collection were found in response t
ext matching regular expression '(?s).*?BusyWorkers:\s([0-9]+).*'
XXXX-XX-XX 14:59:52,315 ERROR [CollectdScheduler-50 Pool-fiber1] HttpCollector: collect: http collection failed: org.opennms.netmgt.collectd.HttpCollector$
HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
org.opennms.netmgt.collectd.HttpCollector$HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
        at org.opennms.netmgt.collectd.HttpCollector.persistResponse(HttpCollector.java:400)
        at org.opennms.netmgt.collectd.HttpCollector.doCollection(HttpCollector.java:253)
        at org.opennms.netmgt.collectd.HttpCollector.access$000(HttpCollector.java:95)
        at org.opennms.netmgt.collectd.HttpCollector$HttpCollectionSet.collect(HttpCollector.java:171)
        at org.opennms.netmgt.collectd.HttpCollector.collect(HttpCollector.java:130)
        at org.opennms.netmgt.collectd.CollectionSpecification.collect(CollectionSpecification.java:223)
        at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:347)
        at org.opennms.netmgt.collectd.CollectableService.run(CollectableService.java:296)
        at org.opennms.netmgt.scheduler.LegacyScheduler$1.run(LegacyScheduler.java:292)

-Oka


Honia A <[hidden email]> wrote:
Hi Oka,
 
Thanks for your reply. I replaced all my code with yours but unfortunately it still doesn't collect the data...
 
On the side note, I used mod_status and got the /server-status to work and the http://192.168.1.171/server-status shows all the data. Then followed the instruction on http://www.opennms.org/index.php/Monitoring_Apache_with_the_HTTP_collector and edited all the config files but still no data is being collected....

Please help :)
 
Thanks

 



Date: Wed, 16 Jul 2008 03:05:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I went and updated the Opennms FAQ page with the changes here so this documented.

-Oka

[hidden email] wrote:
I just added the service "with some minor modifications to not disrupt my system" this works and the .jrb file was created.

My capsd-configuration.xml

     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <property key="url" value="/doccount.html" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

collectd-configuration.xml

    <package name="doc-count">
        <filter>IPADDR != '0.0.0.0'</filter>
        <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
            <parameter key="http-collection" value="doc-count-1" />
            <parameter key="retry" value="1" />
            <parameter key="timeout" value="2000" />
            <parameter key="url" value="/doccount.html"/>
         </service>
     </package>

My http-datacollection-config.xml
  <http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/index.html"
             matches="(?s).*?Document\sCount:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

-Oka

Honia A <[hidden email]> wrote:
Hi Oks,
 
Thanks for your reply. I don't have mod_status running and not sure how to get that to work...I have to do a little research on that first. But for now, as I mentioned before, I am also working on the HttpDocCount example and as you may already know, the regex for this example is pretty simple and the data source is almost one line but the data is not being collected... Can you please do me a favor and take a look at the config files below and see what the problem is?
 
doccount.html (located in /var/www/html direcoty):
 

   <html>
     <head>
       <meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
       <meta name="GENERATOR" content="Microsoft FrontPage 4.0">
       <meta name="ProgId" content="FrontPage.Editor.Document">
       <title>collector test</title>
     </head>
     <body>
       <p>Document Count: 5</p>
     </body>
   </html>

 
capsd-configuration.xml (just the part I added):
 
     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

 
collectd-configuration.xml (just the part I added):
 
  <package name="doc-count">
                 <filter>IPADDR != '0.0.0.0'</filter>
                 <include-range begin="1.1.1.1" end="254.254.254.254"/>

                 <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
                        <parameter key="http-collection" value="doc-count-1" />
                        <parameter key="retry" value="1" />
                        <parameter key="timeout" value="2000" />
                 </service>
        </package>

         <collector service="HttpDocCount" class-name="org.opennms.netmgt.collectd.HttpCollector" />
 
 
http-datacollection-config.xml (the entire file)
 
<http-datacollection-config
    xmlns:http-dc="
http://xmlns.opennms.org/xsd/config/http-datacollection"
    xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
http://xmlns.opennms.org/xsd/config/http-datacollection http://www.opennms.org/xsd/config/http-datacollection-config.xsd"
    rrdRepository="/opt/opennms/share/rrd/snmp/" >
  <http-collection name="weather">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:2016</rra>
      <rra>RRA:AVERAGE:0.5:12:1488</rra>
      <rra>RRA:AVERAGE:0.5:288:366</rra>
      <rra>RRA:MAX:0.5:288:366</rra>
      <rra>RRA:MIN:0.5:288:366</rra>
    </rrd>
    <uris>
      <uri name="weather-station">
        <url path="/httpcolltest.html"
             matches="(?s).*?Temperature.*?3366FF">([0-9\.]+).*?Humidity.*?3366FF">([0-9]+)
                     .*?Barometer.*?3366FF">([0-9\.]+).*?Wind\sChill.*?3366FF"><small>([0-9\.]+)
                     .*?Heat\sIndex.*?3366FF"></small>([0-9\.]+).*"
             response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="wsTemperature" match-group="1" type="gauge32"/>
          <attrib alias="wsHumidity"    match-group="2" type="gauge32"/>
          <attrib alias="wsBarometer"   match-group="3" type="gauge32"/>
          <attrib alias="wsWindChill"   match-group="4" type="gauge32"/>
          <attrib alias="wsHeatIndex"   match-group="5" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

<http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/doccount.html"
             matches="(?s).*Document Count:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>
</http-datacollection-config>

 
poller-configuration.xml (just the part I added):
 
    <service name="HttpDocCount" interval="300000" user-defined="true" status="on">
      <parameter key="retry" value="3" />
      <parameter key="timeout" value="10000" />
      <parameter key="port" value="80" />
      <parameter key="url" value="/" />
      <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
      <parameter key="rrd-base-name" value="httpdoccount" />
      <parameter key="ds-name" value="httpdoccount" />
    </service>

 
 
 <monitor service="HttpDocCount" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor" />
 
 
The HttpDocCount service has been discovered on that node (192.168.1.171) and is being monitored properly. The httpdoccount.jrb has been showed up inside /opt/opennms/share/rrd/response/192.168.1.171 direcory but nothing has ever appeared in /opt/opennms/share/rrd/snmp/192.168.1.171 directory which means no data has been ever collected on this node.
 
The project deadline is approaching and I still could't get any of the examples to work. I am sure since you have already worked on more explicit projects, this should be piece of cake for you. Would that be possible for you to help me figuring this out?
 
Thanks in advance
 



Date: Wed, 16 Jul 2008 01:56:36 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I have just updated the page (as in 10 minutes ago) that Pete linked, it contains copy and paste code.

The reason the "page" you created does not work is that mod_status generates a dynamic page so if you have it running /server-status will work under apache.

Starting with the server-status page would be the easiest place to begin as the regex is very simplistic, the weather page is a complex pattern match and you would have to dig through log files to see what its really saying.

-Oka
 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Need to know now? Get instant answers with Windows Live Messenger. IM on your terms.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Use video conversation to talk face-to-face with Windows Live Messenger. Get started.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


With Windows Live for mobile, your contacts travel with you. Connect on the go.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Stay in touch when you're away with Windows Live Messenger. IM anytime you're online.
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Data Collector example doesn't work

tezarin
In reply to this post by okagau007
Oka,
 
opennms-repo-testing-1.0-12
opennms-core-1.5.92-0.9055.snapshot
opennms-webapp-jetty-1.5.92-0.9055.snapshot
 
Thanks


 



Date: Thu, 17 Jul 2008 00:24:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Which version of NMS are you using?

-Oka

Honia A <[hidden email]> wrote:
Oka,
 
This block of code already exist in my log4j.properties and looks like it's been turned on:
 
# Collectd
log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log
log4j.appender.COLLECTD.layout=org.apache.log4j.PatternLayout
log4j.appender.COLLECTD.layout.ConversionPattern=%d %-5p [%t] %c{1}: %m%n

 
 
About the rescan, I had changed the default value for rescan-frequency in capsd-configuration....it's now 10. I did that so that opennms discover the service immediately...
 
Please advise.
 
Thanks



 



Date: Wed, 16 Jul 2008 22:07:56 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Go into the etc directory and locate the file log4j.properties, and turn on DEBUG for the collectd.log. You shouldn't need to restart NMS for this, force a rescan on the node and then examine the collectd log.

log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log



Honia A <[hidden email]> wrote:
Oka,
 
I looked at the entire log file (collectd.log) but the word "HttpCollector" hasn't been mentioned anywhere.... No Java block either.
 
I use ssh client (Putty) software to connect to the that server which is located on another machine.
 
Please advise.
 
Thanks,
honia
 


 
 



Date: Wed, 16 Jul 2008 05:12:02 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

You will need to paste parts of your collectd.log file search for entries that have this

HttpCollector:

If there is an obvious error you will see a java stack trace associated with it.  Also are you running on the same machine or through a reverse proxy?

This is an example of an error that was generated when there was a misconfigured vhost

XXXX-XX-XX 14:59:52,315 WARN  [CollectdScheduler-50 Pool-fiber1] HttpCollector: doCollection: no attributes defined for collection were found in response t
ext matching regular expression '(?s).*?BusyWorkers:\s([0-9]+).*'
XXXX-XX-XX 14:59:52,315 ERROR [CollectdScheduler-50 Pool-fiber1] HttpCollector: collect: http collection failed: org.opennms.netmgt.collectd.HttpCollector$
HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
org.opennms.netmgt.collectd.HttpCollector$HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
        at org.opennms.netmgt.collectd.HttpCollector.persistResponse(HttpCollector.java:400)
        at org.opennms.netmgt.collectd.HttpCollector.doCollection(HttpCollector.java:253)
        at org.opennms.netmgt.collectd.HttpCollector.access$000(HttpCollector.java:95)
        at org.opennms.netmgt.collectd.HttpCollector$HttpCollectionSet.collect(HttpCollector.java:171)
        at org.opennms.netmgt.collectd.HttpCollector.collect(HttpCollector.java:130)
        at org.opennms.netmgt.collectd.CollectionSpecification.collect(CollectionSpecification.java:223)
        at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:347)
        at org.opennms.netmgt.collectd.CollectableService.run(CollectableService.java:296)
        at org.opennms.netmgt.scheduler.LegacyScheduler$1.run(LegacyScheduler.java:292)

-Oka


Honia A <[hidden email]> wrote:
Hi Oka,
 
Thanks for your reply. I replaced all my code with yours but unfortunately it still doesn't collect the data...
 
On the side note, I used mod_status and got the /server-status to work and the http://192.168.1.171/server-status shows all the data. Then followed the instruction on http://www.opennms.org/index.php/Monitoring_Apache_with_the_HTTP_collector and edited all the config files but still no data is being collected....

Please help :)
 
Thanks

 



Date: Wed, 16 Jul 2008 03:05:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I went and updated the Opennms FAQ page with the changes here so this documented.

-Oka

[hidden email] wrote:
I just added the service "with some minor modifications to not disrupt my system" this works and the .jrb file was created.

My capsd-configuration.xml

     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <property key="url" value="/doccount.html" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

collectd-configuration.xml

    <package name="doc-count">
        <filter>IPADDR != '0.0.0.0'</filter>
        <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
            <parameter key="http-collection" value="doc-count-1" />
            <parameter key="retry" value="1" />
            <parameter key="timeout" value="2000" />
            <parameter key="url" value="/doccount.html"/>
         </service>
     </package>

My http-datacollection-config.xml
  <http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/index.html"
             matches="(?s).*?Document\sCount:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

-Oka

Honia A <[hidden email]> wrote:
Hi Oks,
 
Thanks for your reply. I don't have mod_status running and not sure how to get that to work...I have to do a little research on that first. But for now, as I mentioned before, I am also working on the HttpDocCount example and as you may already know, the regex for this example is pretty simple and the data source is almost one line but the data is not being collected... Can you please do me a favor and take a look at the config files below and see what the problem is?
 
doccount.html (located in /var/www/html direcoty):
 

   <html>
     <head>
       <meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
       <meta name="GENERATOR" content="Microsoft FrontPage 4.0">
       <meta name="ProgId" content="FrontPage.Editor.Document">
       <title>collector test</title>
     </head>
     <body>
       <p>Document Count: 5</p>
     </body>
   </html>

 
capsd-configuration.xml (just the part I added):
 
     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

 
collectd-configuration.xml (just the part I added):
 
  <package name="doc-count">
                 <filter>IPADDR != '0.0.0.0'</filter>
                 <include-range begin="1.1.1.1" end="254.254.254.254"/>

                 <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
                        <parameter key="http-collection" value="doc-count-1" />
                        <parameter key="retry" value="1" />
                        <parameter key="timeout" value="2000" />
                 </service>
        </package>

         <collector service="HttpDocCount" class-name="org.opennms.netmgt.collectd.HttpCollector" />
 
 
http-datacollection-config.xml (the entire file)
 
<http-datacollection-config
    xmlns:http-dc="
http://xmlns.opennms.org/xsd/config/http-datacollection"
    xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
http://xmlns.opennms.org/xsd/config/http-datacollection http://www.opennms.org/xsd/config/http-datacollection-config.xsd"
    rrdRepository="/opt/opennms/share/rrd/snmp/" >
  <http-collection name="weather">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:2016</rra>
      <rra>RRA:AVERAGE:0.5:12:1488</rra>
      <rra>RRA:AVERAGE:0.5:288:366</rra>
      <rra>RRA:MAX:0.5:288:366</rra>
      <rra>RRA:MIN:0.5:288:366</rra>
    </rrd>
    <uris>
      <uri name="weather-station">
        <url path="/httpcolltest.html"
             matches="(?s).*?Temperature.*?3366FF">([0-9\.]+).*?Humidity.*?3366FF">([0-9]+)
                     .*?Barometer.*?3366FF">([0-9\.]+).*?Wind\sChill.*?3366FF"><small>([0-9\.]+)
                     .*?Heat\sIndex.*?3366FF"></small>([0-9\.]+).*"
             response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="wsTemperature" match-group="1" type="gauge32"/>
          <attrib alias="wsHumidity"    match-group="2" type="gauge32"/>
          <attrib alias="wsBarometer"   match-group="3" type="gauge32"/>
          <attrib alias="wsWindChill"   match-group="4" type="gauge32"/>
          <attrib alias="wsHeatIndex"   match-group="5" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

<http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/doccount.html"
             matches="(?s).*Document Count:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>
</http-datacollection-config>

 
poller-configuration.xml (just the part I added):
 
    <service name="HttpDocCount" interval="300000" user-defined="true" status="on">
      <parameter key="retry" value="3" />
      <parameter key="timeout" value="10000" />
      <parameter key="port" value="80" />
      <parameter key="url" value="/" />
      <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
      <parameter key="rrd-base-name" value="httpdoccount" />
      <parameter key="ds-name" value="httpdoccount" />
    </service>

 
 
 <monitor service="HttpDocCount" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor" />
 
 
The HttpDocCount service has been discovered on that node (192.168.1.171) and is being monitored properly. The httpdoccount.jrb has been showed up inside /opt/opennms/share/rrd/response/192.168.1.171 direcory but nothing has ever appeared in /opt/opennms/share/rrd/snmp/192.168.1.171 directory which means no data has been ever collected on this node.
 
The project deadline is approaching and I still could't get any of the examples to work. I am sure since you have already worked on more explicit projects, this should be piece of cake for you. Would that be possible for you to help me figuring this out?
 
Thanks in advance
 



Date: Wed, 16 Jul 2008 01:56:36 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I have just updated the page (as in 10 minutes ago) that Pete linked, it contains copy and paste code.

The reason the "page" you created does not work is that mod_status generates a dynamic page so if you have it running /server-status will work under apache.

Starting with the server-status page would be the easiest place to begin as the regex is very simplistic, the weather page is a complex pattern match and you would have to dig through log files to see what its really saying.

-Oka
 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Need to know now? Get instant answers with Windows Live Messenger. IM on your terms.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Use video conversation to talk face-to-face with Windows Live Messenger. Get started.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


With Windows Live for mobile, your contacts travel with you. Connect on the go.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Time for vacation? WIN what you need. Enter Now!-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


With Windows Live for mobile, your contacts travel with you. Connect on the go.
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Data Collector example doesn't work

tezarin
In reply to this post by okagau007
OK I think I know what the problem is...
 
At the top of the http-datacollection-config.xml file there are some xml links which need to be updated...some of the links are broken. I'll report back.


 



Date: Thu, 17 Jul 2008 00:24:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Which version of NMS are you using?

-Oka

Honia A <[hidden email]> wrote:
Oka,
 
This block of code already exist in my log4j.properties and looks like it's been turned on:
 
# Collectd
log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log
log4j.appender.COLLECTD.layout=org.apache.log4j.PatternLayout
log4j.appender.COLLECTD.layout.ConversionPattern=%d %-5p [%t] %c{1}: %m%n

 
 
About the rescan, I had changed the default value for rescan-frequency in capsd-configuration....it's now 10. I did that so that opennms discover the service immediately...
 
Please advise.
 
Thanks



 



Date: Wed, 16 Jul 2008 22:07:56 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Go into the etc directory and locate the file log4j.properties, and turn on DEBUG for the collectd.log. You shouldn't need to restart NMS for this, force a rescan on the node and then examine the collectd log.

log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log



Honia A <[hidden email]> wrote:
Oka,
 
I looked at the entire log file (collectd.log) but the word "HttpCollector" hasn't been mentioned anywhere.... No Java block either.
 
I use ssh client (Putty) software to connect to the that server which is located on another machine.
 
Please advise.
 
Thanks,
honia
 


 
 



Date: Wed, 16 Jul 2008 05:12:02 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

You will need to paste parts of your collectd.log file search for entries that have this

HttpCollector:

If there is an obvious error you will see a java stack trace associated with it.  Also are you running on the same machine or through a reverse proxy?

This is an example of an error that was generated when there was a misconfigured vhost

XXXX-XX-XX 14:59:52,315 WARN  [CollectdScheduler-50 Pool-fiber1] HttpCollector: doCollection: no attributes defined for collection were found in response t
ext matching regular expression '(?s).*?BusyWorkers:\s([0-9]+).*'
XXXX-XX-XX 14:59:52,315 ERROR [CollectdScheduler-50 Pool-fiber1] HttpCollector: collect: http collection failed: org.opennms.netmgt.collectd.HttpCollector$
HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
org.opennms.netmgt.collectd.HttpCollector$HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
        at org.opennms.netmgt.collectd.HttpCollector.persistResponse(HttpCollector.java:400)
        at org.opennms.netmgt.collectd.HttpCollector.doCollection(HttpCollector.java:253)
        at org.opennms.netmgt.collectd.HttpCollector.access$000(HttpCollector.java:95)
        at org.opennms.netmgt.collectd.HttpCollector$HttpCollectionSet.collect(HttpCollector.java:171)
        at org.opennms.netmgt.collectd.HttpCollector.collect(HttpCollector.java:130)
        at org.opennms.netmgt.collectd.CollectionSpecification.collect(CollectionSpecification.java:223)
        at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:347)
        at org.opennms.netmgt.collectd.CollectableService.run(CollectableService.java:296)
        at org.opennms.netmgt.scheduler.LegacyScheduler$1.run(LegacyScheduler.java:292)

-Oka


Honia A <[hidden email]> wrote:
Hi Oka,
 
Thanks for your reply. I replaced all my code with yours but unfortunately it still doesn't collect the data...
 
On the side note, I used mod_status and got the /server-status to work and the http://192.168.1.171/server-status shows all the data. Then followed the instruction on http://www.opennms.org/index.php/Monitoring_Apache_with_the_HTTP_collector and edited all the config files but still no data is being collected....

Please help :)
 
Thanks

 



Date: Wed, 16 Jul 2008 03:05:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I went and updated the Opennms FAQ page with the changes here so this documented.

-Oka

[hidden email] wrote:
I just added the service "with some minor modifications to not disrupt my system" this works and the .jrb file was created.

My capsd-configuration.xml

     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <property key="url" value="/doccount.html" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

collectd-configuration.xml

    <package name="doc-count">
        <filter>IPADDR != '0.0.0.0'</filter>
        <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
            <parameter key="http-collection" value="doc-count-1" />
            <parameter key="retry" value="1" />
            <parameter key="timeout" value="2000" />
            <parameter key="url" value="/doccount.html"/>
         </service>
     </package>

My http-datacollection-config.xml
  <http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/index.html"
             matches="(?s).*?Document\sCount:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

-Oka

Honia A <[hidden email]> wrote:
Hi Oks,
 
Thanks for your reply. I don't have mod_status running and not sure how to get that to work...I have to do a little research on that first. But for now, as I mentioned before, I am also working on the HttpDocCount example and as you may already know, the regex for this example is pretty simple and the data source is almost one line but the data is not being collected... Can you please do me a favor and take a look at the config files below and see what the problem is?
 
doccount.html (located in /var/www/html direcoty):
 

   <html>
     <head>
       <meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
       <meta name="GENERATOR" content="Microsoft FrontPage 4.0">
       <meta name="ProgId" content="FrontPage.Editor.Document">
       <title>collector test</title>
     </head>
     <body>
       <p>Document Count: 5</p>
     </body>
   </html>

 
capsd-configuration.xml (just the part I added):
 
     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

 
collectd-configuration.xml (just the part I added):
 
  <package name="doc-count">
                 <filter>IPADDR != '0.0.0.0'</filter>
                 <include-range begin="1.1.1.1" end="254.254.254.254"/>

                 <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
                        <parameter key="http-collection" value="doc-count-1" />
                        <parameter key="retry" value="1" />
                        <parameter key="timeout" value="2000" />
                 </service>
        </package>

         <collector service="HttpDocCount" class-name="org.opennms.netmgt.collectd.HttpCollector" />
 
 
http-datacollection-config.xml (the entire file)
 
<http-datacollection-config
    xmlns:http-dc="
http://xmlns.opennms.org/xsd/config/http-datacollection"
    xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
http://xmlns.opennms.org/xsd/config/http-datacollection http://www.opennms.org/xsd/config/http-datacollection-config.xsd"
    rrdRepository="/opt/opennms/share/rrd/snmp/" >
  <http-collection name="weather">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:2016</rra>
      <rra>RRA:AVERAGE:0.5:12:1488</rra>
      <rra>RRA:AVERAGE:0.5:288:366</rra>
      <rra>RRA:MAX:0.5:288:366</rra>
      <rra>RRA:MIN:0.5:288:366</rra>
    </rrd>
    <uris>
      <uri name="weather-station">
        <url path="/httpcolltest.html"
             matches="(?s).*?Temperature.*?3366FF">([0-9\.]+).*?Humidity.*?3366FF">([0-9]+)
                     .*?Barometer.*?3366FF">([0-9\.]+).*?Wind\sChill.*?3366FF"><small>([0-9\.]+)
                     .*?Heat\sIndex.*?3366FF"></small>([0-9\.]+).*"
             response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="wsTemperature" match-group="1" type="gauge32"/>
          <attrib alias="wsHumidity"    match-group="2" type="gauge32"/>
          <attrib alias="wsBarometer"   match-group="3" type="gauge32"/>
          <attrib alias="wsWindChill"   match-group="4" type="gauge32"/>
          <attrib alias="wsHeatIndex"   match-group="5" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

<http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/doccount.html"
             matches="(?s).*Document Count:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>
</http-datacollection-config>

 
poller-configuration.xml (just the part I added):
 
    <service name="HttpDocCount" interval="300000" user-defined="true" status="on">
      <parameter key="retry" value="3" />
      <parameter key="timeout" value="10000" />
      <parameter key="port" value="80" />
      <parameter key="url" value="/" />
      <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
      <parameter key="rrd-base-name" value="httpdoccount" />
      <parameter key="ds-name" value="httpdoccount" />
    </service>

 
 
 <monitor service="HttpDocCount" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor" />
 
 
The HttpDocCount service has been discovered on that node (192.168.1.171) and is being monitored properly. The httpdoccount.jrb has been showed up inside /opt/opennms/share/rrd/response/192.168.1.171 direcory but nothing has ever appeared in /opt/opennms/share/rrd/snmp/192.168.1.171 directory which means no data has been ever collected on this node.
 
The project deadline is approaching and I still could't get any of the examples to work. I am sure since you have already worked on more explicit projects, this should be piece of cake for you. Would that be possible for you to help me figuring this out?
 
Thanks in advance
 



Date: Wed, 16 Jul 2008 01:56:36 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I have just updated the page (as in 10 minutes ago) that Pete linked, it contains copy and paste code.

The reason the "page" you created does not work is that mod_status generates a dynamic page so if you have it running /server-status will work under apache.

Starting with the server-status page would be the easiest place to begin as the regex is very simplistic, the weather page is a complex pattern match and you would have to dig through log files to see what its really saying.

-Oka
 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Need to know now? Get instant answers with Windows Live Messenger. IM on your terms.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Use video conversation to talk face-to-face with Windows Live Messenger. Get started.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


With Windows Live for mobile, your contacts travel with you. Connect on the go.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Time for vacation? WIN what you need. Enter Now!-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


With Windows Live for mobile, your contacts travel with you. Connect on the go.
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Data Collector example doesn't work

tezarin
In reply to this post by okagau007
Ans I am going to remove the HttpWeatherStation as the error clearly complains about "small" which belongs to this service's Regex...I'll focus on Apache-Stats and HttpDocCount for now ;)

 



Date: Thu, 17 Jul 2008 00:24:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Which version of NMS are you using?

-Oka

Honia A <[hidden email]> wrote:
Oka,
 
This block of code already exist in my log4j.properties and looks like it's been turned on:
 
# Collectd
log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log
log4j.appender.COLLECTD.layout=org.apache.log4j.PatternLayout
log4j.appender.COLLECTD.layout.ConversionPattern=%d %-5p [%t] %c{1}: %m%n

 
 
About the rescan, I had changed the default value for rescan-frequency in capsd-configuration....it's now 10. I did that so that opennms discover the service immediately...
 
Please advise.
 
Thanks



 



Date: Wed, 16 Jul 2008 22:07:56 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Go into the etc directory and locate the file log4j.properties, and turn on DEBUG for the collectd.log. You shouldn't need to restart NMS for this, force a rescan on the node and then examine the collectd log.

log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log



Honia A <[hidden email]> wrote:
Oka,
 
I looked at the entire log file (collectd.log) but the word "HttpCollector" hasn't been mentioned anywhere.... No Java block either.
 
I use ssh client (Putty) software to connect to the that server which is located on another machine.
 
Please advise.
 
Thanks,
honia
 


 
 



Date: Wed, 16 Jul 2008 05:12:02 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

You will need to paste parts of your collectd.log file search for entries that have this

HttpCollector:

If there is an obvious error you will see a java stack trace associated with it.  Also are you running on the same machine or through a reverse proxy?

This is an example of an error that was generated when there was a misconfigured vhost

XXXX-XX-XX 14:59:52,315 WARN  [CollectdScheduler-50 Pool-fiber1] HttpCollector: doCollection: no attributes defined for collection were found in response t
ext matching regular expression '(?s).*?BusyWorkers:\s([0-9]+).*'
XXXX-XX-XX 14:59:52,315 ERROR [CollectdScheduler-50 Pool-fiber1] HttpCollector: collect: http collection failed: org.opennms.netmgt.collectd.HttpCollector$
HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
org.opennms.netmgt.collectd.HttpCollector$HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
        at org.opennms.netmgt.collectd.HttpCollector.persistResponse(HttpCollector.java:400)
        at org.opennms.netmgt.collectd.HttpCollector.doCollection(HttpCollector.java:253)
        at org.opennms.netmgt.collectd.HttpCollector.access$000(HttpCollector.java:95)
        at org.opennms.netmgt.collectd.HttpCollector$HttpCollectionSet.collect(HttpCollector.java:171)
        at org.opennms.netmgt.collectd.HttpCollector.collect(HttpCollector.java:130)
        at org.opennms.netmgt.collectd.CollectionSpecification.collect(CollectionSpecification.java:223)
        at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:347)
        at org.opennms.netmgt.collectd.CollectableService.run(CollectableService.java:296)
        at org.opennms.netmgt.scheduler.LegacyScheduler$1.run(LegacyScheduler.java:292)

-Oka


Honia A <[hidden email]> wrote:
Hi Oka,
 
Thanks for your reply. I replaced all my code with yours but unfortunately it still doesn't collect the data...
 
On the side note, I used mod_status and got the /server-status to work and the http://192.168.1.171/server-status shows all the data. Then followed the instruction on http://www.opennms.org/index.php/Monitoring_Apache_with_the_HTTP_collector and edited all the config files but still no data is being collected....

Please help :)
 
Thanks

 



Date: Wed, 16 Jul 2008 03:05:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I went and updated the Opennms FAQ page with the changes here so this documented.

-Oka

[hidden email] wrote:
I just added the service "with some minor modifications to not disrupt my system" this works and the .jrb file was created.

My capsd-configuration.xml

     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <property key="url" value="/doccount.html" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

collectd-configuration.xml

    <package name="doc-count">
        <filter>IPADDR != '0.0.0.0'</filter>
        <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
            <parameter key="http-collection" value="doc-count-1" />
            <parameter key="retry" value="1" />
            <parameter key="timeout" value="2000" />
            <parameter key="url" value="/doccount.html"/>
         </service>
     </package>

My http-datacollection-config.xml
  <http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/index.html"
             matches="(?s).*?Document\sCount:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

-Oka

Honia A <[hidden email]> wrote:
Hi Oks,
 
Thanks for your reply. I don't have mod_status running and not sure how to get that to work...I have to do a little research on that first. But for now, as I mentioned before, I am also working on the HttpDocCount example and as you may already know, the regex for this example is pretty simple and the data source is almost one line but the data is not being collected... Can you please do me a favor and take a look at the config files below and see what the problem is?
 
doccount.html (located in /var/www/html direcoty):
 

   <html>
     <head>
       <meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
       <meta name="GENERATOR" content="Microsoft FrontPage 4.0">
       <meta name="ProgId" content="FrontPage.Editor.Document">
       <title>collector test</title>
     </head>
     <body>
       <p>Document Count: 5</p>
     </body>
   </html>

 
capsd-configuration.xml (just the part I added):
 
     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

 
collectd-configuration.xml (just the part I added):
 
  <package name="doc-count">
                 <filter>IPADDR != '0.0.0.0'</filter>
                 <include-range begin="1.1.1.1" end="254.254.254.254"/>

                 <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
                        <parameter key="http-collection" value="doc-count-1" />
                        <parameter key="retry" value="1" />
                        <parameter key="timeout" value="2000" />
                 </service>
        </package>

         <collector service="HttpDocCount" class-name="org.opennms.netmgt.collectd.HttpCollector" />
 
 
http-datacollection-config.xml (the entire file)
 
<http-datacollection-config
    xmlns:http-dc="
http://xmlns.opennms.org/xsd/config/http-datacollection"
    xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
http://xmlns.opennms.org/xsd/config/http-datacollection http://www.opennms.org/xsd/config/http-datacollection-config.xsd"
    rrdRepository="/opt/opennms/share/rrd/snmp/" >
  <http-collection name="weather">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:2016</rra>
      <rra>RRA:AVERAGE:0.5:12:1488</rra>
      <rra>RRA:AVERAGE:0.5:288:366</rra>
      <rra>RRA:MAX:0.5:288:366</rra>
      <rra>RRA:MIN:0.5:288:366</rra>
    </rrd>
    <uris>
      <uri name="weather-station">
        <url path="/httpcolltest.html"
             matches="(?s).*?Temperature.*?3366FF">([0-9\.]+).*?Humidity.*?3366FF">([0-9]+)
                     .*?Barometer.*?3366FF">([0-9\.]+).*?Wind\sChill.*?3366FF"><small>([0-9\.]+)
                     .*?Heat\sIndex.*?3366FF"></small>([0-9\.]+).*"
             response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="wsTemperature" match-group="1" type="gauge32"/>
          <attrib alias="wsHumidity"    match-group="2" type="gauge32"/>
          <attrib alias="wsBarometer"   match-group="3" type="gauge32"/>
          <attrib alias="wsWindChill"   match-group="4" type="gauge32"/>
          <attrib alias="wsHeatIndex"   match-group="5" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

<http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/doccount.html"
             matches="(?s).*Document Count:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>
</http-datacollection-config>

 
poller-configuration.xml (just the part I added):
 
    <service name="HttpDocCount" interval="300000" user-defined="true" status="on">
      <parameter key="retry" value="3" />
      <parameter key="timeout" value="10000" />
      <parameter key="port" value="80" />
      <parameter key="url" value="/" />
      <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
      <parameter key="rrd-base-name" value="httpdoccount" />
      <parameter key="ds-name" value="httpdoccount" />
    </service>

 
 
 <monitor service="HttpDocCount" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor" />
 
 
The HttpDocCount service has been discovered on that node (192.168.1.171) and is being monitored properly. The httpdoccount.jrb has been showed up inside /opt/opennms/share/rrd/response/192.168.1.171 direcory but nothing has ever appeared in /opt/opennms/share/rrd/snmp/192.168.1.171 directory which means no data has been ever collected on this node.
 
The project deadline is approaching and I still could't get any of the examples to work. I am sure since you have already worked on more explicit projects, this should be piece of cake for you. Would that be possible for you to help me figuring this out?
 
Thanks in advance
 



Date: Wed, 16 Jul 2008 01:56:36 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I have just updated the page (as in 10 minutes ago) that Pete linked, it contains copy and paste code.

The reason the "page" you created does not work is that mod_status generates a dynamic page so if you have it running /server-status will work under apache.

Starting with the server-status page would be the easiest place to begin as the regex is very simplistic, the weather page is a complex pattern match and you would have to dig through log files to see what its really saying.

-Oka
 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Need to know now? Get instant answers with Windows Live Messenger. IM on your terms.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Use video conversation to talk face-to-face with Windows Live Messenger. Get started.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


With Windows Live for mobile, your contacts travel with you. Connect on the go.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Time for vacation? WIN what you need. Enter Now!-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Use video conversation to talk face-to-face with Windows Live Messenger. Get started.
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Start up java error

Mudasar
In reply to this post by tezarin

Hi,

 

I got the following error when I try to restart the opennms, i think it is due to less open connection of the database, if not please let me know how can I solve it because I also search to increase db connection of postgres but not find.

 

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

 

org.springframework.jdbc.UncategorizedSQLException: Hibernate operation: could not load an entity: [org.opennms.netmgt.model.OnmsIpInterface#733]; uncategorized SQLException for SQL [select onmsipinte0_.id as id4_0_, onmsipinte0_.nodeId as nodeId4_0_, onmsipinte0_.ipAddr as ipAddr4_0_, onmsipinte0_.isSnmpPrimary as isSnmpPr3_4_0_, onmsipinte0_.ipHostName as ipHostName4_0_, onmsipinte0_.isManaged as isManaged4_0_, onmsipinte0_.ipStatus as ipStatus4_0_, onmsipinte0_.ipLastCapsdPoll as ipLastCa7_4_0_, onmsipinte0_.snmpInterfaceId as snmpInte9_4_0_ from ipInterface onmsipinte0_ where onmsipinte0_.id=?]; SQL state [null]; error code [0]; Connections could not be acquired from the underlying database!; nested exception is java.sql.SQLException: Connections could not be acquired from the underlying database!

Caused by: java.sql.SQLException: Connections could not be acquired from the underlying database!

        at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:104)

        at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:264)

        at com.mchange.v2.c3p0.PoolBackedDataSource.getConnection(PoolBackedDataSource.java:94)

        at com.mchange.v2.c3p0.ComboPooledDataSource.getConnection(ComboPooledDataSource.java:521)

        at org.opennms.netmgt.config.C3P0ConnectionFactory.getConnection(C3P0ConnectionFactory.java:171)

        at org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy$LazyConnectionInvocationHandler.getTargetConnection(LazyConnectionDataSourceProxy.java:363)

        at org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy$LazyConnectionInvocationHandler.invoke(LazyConnectionDataSourceProxy.java:338)

        at $Proxy27.prepareStatement(Unknown Source)

        at org.hibernate.jdbc.AbstractBatcher.getPreparedStatement(AbstractBatcher.java:505)

        at org.hibernate.jdbc.AbstractBatcher.getPreparedStatement(AbstractBatcher.java:423)

        at org.hibernate.jdbc.AbstractBatcher.prepareQueryStatement(AbstractBatcher.java:139)

        at org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:1547)

        at org.hibernate.loader.Loader.doQuery(Loader.java:673)

        at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:236)

        at org.hibernate.loader.Loader.loadEntity(Loader.java:1860)

        at org.hibernate.loader.entity.AbstractEntityLoader.load(AbstractEntityLoader.java:48)

        at org.hibernate.loader.entity.AbstractEntityLoader.load(AbstractEntityLoader.java:42)

        at org.hibernate.persister.entity.AbstractEntityPersister.load(AbstractEntityPersister.java:3044)

        at org.hibernate.event.def.DefaultLoadEventListener.loadFromDatasource(DefaultLoadEventListener.java:395)

        at org.hibernate.event.def.DefaultLoadEventListener.doLoad(DefaultLoadEventListener.java:375)

        at org.hibernate.event.def.DefaultLoadEventListener.load(DefaultLoadEventListener.java:139)

        at org.hibernate.event.def.DefaultLoadEventListener.proxyOrLoad(DefaultLoadEventListener.java:195)

        at org.hibernate.event.def.DefaultLoadEventListener.onLoad(DefaultLoadEventListener.java:103)

        at org.hibernate.impl.SessionImpl.fireLoad(SessionImpl.java:878)

        at org.hibernate.impl.SessionImpl.get(SessionImpl.java:815)

        at org.hibernate.impl.SessionImpl.get(SessionImpl.java:808)

        at org.springframework.orm.hibernate3.HibernateTemplate$1.doInHibernate(HibernateTemplate.java:470)

        at org.springframework.orm.hibernate3.HibernateTemplate.execute(HibernateTemplate.java:372)

        at org.springframework.orm.hibernate3.HibernateTemplate.get(HibernateTemplate.java:464)

        at org.springframework.orm.hibernate3.HibernateTemplate.get(HibernateTemplate.java:458)

        at org.opennms.netmgt.dao.hibernate.AbstractDaoHibernate.get(AbstractDaoHibernate.java:195)

        at org.opennms.netmgt.collectd.DefaultCollectionAgentService.getIpInterface(DefaultCollectionAgentService.java:99)

        at org.opennms.netmgt.collectd.DefaultCollectionAgentService.getNode(DefaultCollectionAgentService.java:103)

        at org.opennms.netmgt.collectd.DefaultCollectionAgentService.getSnmpInterfaces(DefaultCollectionAgentService.java:169)

        at org.opennms.netmgt.collectd.DefaultCollectionAgentService.getSnmpInterfaceData(DefaultCollectionAgentService.java:154)

        at sun.reflect.GeneratedMethodAccessor216.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

        at java.lang.reflect.Method.invoke(Method.java:597)

        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:304)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)

        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)

        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)

        at $Proxy43.getSnmpInterfaceData(Unknown Source)

        at org.opennms.netmgt.collectd.DefaultCollectionAgent.getSnmpInterfaceData(DefaultCollectionAgent.java:260)

        at org.opennms.netmgt.collectd.DefaultCollectionAgent.getSnmpInterfaceInfo(DefaultCollectionAgent.java:270)

        at org.opennms.netmgt.collectd.IfResourceType.addKnownIfResources(IfResourceType.java:65)        at org.opennms.netmgt.collectd.IfResourceType.<init>(IfResourceType.java:53)        at org.opennms.netmgt.collectd.OnmsSnmpCollection.getIfResourceType(OnmsSnmpCollection.java:238)        at org.opennms.netmgt.collectd.SnmpCollectionSet.getIfResourceType(SnmpCollectionSet.java:445)        at org.opennms.netmgt.collectd.SnmpCollectionSet.hasInterfaceDataToCollect(SnmpCollectionSet.java:180)

        at org.opennms.netmgt.collectd.SnmpCollectionSet.createIfNumberTracker(SnmpCollectionSet.java:148)

        at org.opennms.netmgt.collectd.SnmpCollectionSet.getIfNumber(SnmpCollectionSet.java:122)

        at org.opennms.netmgt.collectd.SnmpCollectionSet.getTracker(SnmpCollectionSet.java:243)

        at org.opennms.netmgt.collectd.SnmpCollectionSet.createWalker(SnmpCollectionSet.java:261)

        at org.opennms.netmgt.collectd.SnmpCollectionSet.collect(SnmpCollectionSet.java:292)

        at org.opennms.netmgt.collectd.SnmpCollector.collect(SnmpCollector.java:395)

        at org.opennms.netmgt.collectd.CollectionSpecification.collect(CollectionSpecification.java:226)

        at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:346)

        at org.opennms.netmgt.collectd.CollectableService.run(CollectableService.java:295)

        at org.opennms.netmgt.scheduler.LegacyScheduler$1.run(LegacyScheduler.java:292)

        at org.opennms.core.concurrent.RunnableConsumerThreadPool$FiberThreadImpl.run(RunnableConsumerThreadPool.java:422)

        at java.lang.Thread.run(Thread.java:619)

Caused by: com.mchange.v2.resourcepool.CannotAcquireResourceException: A ResourcePool could not acquire a resource from its primary factory or source.

        at com.mchange.v2.resourcepool.BasicResourcePool.awaitAcquire(BasicResourcePool.java:972)

        at com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResourcePool.java:208)

        at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:260)

        ... 62 more

 

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

 

 

Thanks

 

Mudasar


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Data Collector example doesn't work

tezarin
In reply to this post by okagau007
Good news! The data is being collected now! Here are the list of the problems which I was able to find and solve:
 
1) The xml links were all incorrect...so I replaced them with the one I found on ONMS website.
 
2) Removed the HttpWeatherStation service and all related blocks of code from all the files. The value "small" was causing the data collection process to crash.
 
3) Found duplicate files: collectd-configuration.xml, collectd-configuration.xml~ The second one was empty (just the default info), so I copied/pasted the lines from file 1 to file 2. So until I figure out which one is the real one, I will keep both.
 
Now, inside /opt/opennms/share/rrd/snmp/ directory, a new folder was created called "23" which has the following files in it:
 
BusyWorkers.jrb, BytesPerSec.jrb, documentCount.jrb and IdleWorkers.jrb.
 
Now I was wondering if someone could tell me:
 
1 - How to see the content of these files (I understand it's a jrobin file and I probably have the jar file on my system)
 
2 - Why the folder is called 23?
 
3 - Shouldn't this new folder be created inside the /opt/opennms/share/rrd/snmp/192.168.1.171 directory? (I thuoght I added these service to this IP)...
 
Thanks again,
Honia

 
 


 
 



Date: Thu, 17 Jul 2008 00:24:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Which version of NMS are you using?

-Oka

Honia A <[hidden email]> wrote:
Oka,
 
This block of code already exist in my log4j.properties and looks like it's been turned on:
 
# Collectd
log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log
log4j.appender.COLLECTD.layout=org.apache.log4j.PatternLayout
log4j.appender.COLLECTD.layout.ConversionPattern=%d %-5p [%t] %c{1}: %m%n

 
 
About the rescan, I had changed the default value for rescan-frequency in capsd-configuration....it's now 10. I did that so that opennms discover the service immediately...
 
Please advise.
 
Thanks



 



Date: Wed, 16 Jul 2008 22:07:56 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Go into the etc directory and locate the file log4j.properties, and turn on DEBUG for the collectd.log. You shouldn't need to restart NMS for this, force a rescan on the node and then examine the collectd log.

log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log



Honia A <[hidden email]> wrote:
Oka,
 
I looked at the entire log file (collectd.log) but the word "HttpCollector" hasn't been mentioned anywhere.... No Java block either.
 
I use ssh client (Putty) software to connect to the that server which is located on another machine.
 
Please advise.
 
Thanks,
honia
 


 
 



Date: Wed, 16 Jul 2008 05:12:02 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

You will need to paste parts of your collectd.log file search for entries that have this

HttpCollector:

If there is an obvious error you will see a java stack trace associated with it.  Also are you running on the same machine or through a reverse proxy?

This is an example of an error that was generated when there was a misconfigured vhost

XXXX-XX-XX 14:59:52,315 WARN  [CollectdScheduler-50 Pool-fiber1] HttpCollector: doCollection: no attributes defined for collection were found in response t
ext matching regular expression '(?s).*?BusyWorkers:\s([0-9]+).*'
XXXX-XX-XX 14:59:52,315 ERROR [CollectdScheduler-50 Pool-fiber1] HttpCollector: collect: http collection failed: org.opennms.netmgt.collectd.HttpCollector$
HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
org.opennms.netmgt.collectd.HttpCollector$HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
        at org.opennms.netmgt.collectd.HttpCollector.persistResponse(HttpCollector.java:400)
        at org.opennms.netmgt.collectd.HttpCollector.doCollection(HttpCollector.java:253)
        at org.opennms.netmgt.collectd.HttpCollector.access$000(HttpCollector.java:95)
        at org.opennms.netmgt.collectd.HttpCollector$HttpCollectionSet.collect(HttpCollector.java:171)
        at org.opennms.netmgt.collectd.HttpCollector.collect(HttpCollector.java:130)
        at org.opennms.netmgt.collectd.CollectionSpecification.collect(CollectionSpecification.java:223)
        at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:347)
        at org.opennms.netmgt.collectd.CollectableService.run(CollectableService.java:296)
        at org.opennms.netmgt.scheduler.LegacyScheduler$1.run(LegacyScheduler.java:292)

-Oka


Honia A <[hidden email]> wrote:
Hi Oka,
 
Thanks for your reply. I replaced all my code with yours but unfortunately it still doesn't collect the data...
 
On the side note, I used mod_status and got the /server-status to work and the http://192.168.1.171/server-status shows all the data. Then followed the instruction on http://www.opennms.org/index.php/Monitoring_Apache_with_the_HTTP_collector and edited all the config files but still no data is being collected....

Please help :)
 
Thanks

 



Date: Wed, 16 Jul 2008 03:05:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I went and updated the Opennms FAQ page with the changes here so this documented.

-Oka

[hidden email] wrote:
I just added the service "with some minor modifications to not disrupt my system" this works and the .jrb file was created.

My capsd-configuration.xml

     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <property key="url" value="/doccount.html" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

collectd-configuration.xml

    <package name="doc-count">
        <filter>IPADDR != '0.0.0.0'</filter>
        <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
            <parameter key="http-collection" value="doc-count-1" />
            <parameter key="retry" value="1" />
            <parameter key="timeout" value="2000" />
            <parameter key="url" value="/doccount.html"/>
         </service>
     </package>

My http-datacollection-config.xml
  <http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/index.html"
             matches="(?s).*?Document\sCount:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

-Oka

Honia A <[hidden email]> wrote:
Hi Oks,
 
Thanks for your reply. I don't have mod_status running and not sure how to get that to work...I have to do a little research on that first. But for now, as I mentioned before, I am also working on the HttpDocCount example and as you may already know, the regex for this example is pretty simple and the data source is almost one line but the data is not being collected... Can you please do me a favor and take a look at the config files below and see what the problem is?
 
doccount.html (located in /var/www/html direcoty):
 

   <html>
     <head>
       <meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
       <meta name="GENERATOR" content="Microsoft FrontPage 4.0">
       <meta name="ProgId" content="FrontPage.Editor.Document">
       <title>collector test</title>
     </head>
     <body>
       <p>Document Count: 5</p>
     </body>
   </html>

 
capsd-configuration.xml (just the part I added):
 
     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

 
collectd-configuration.xml (just the part I added):
 
  <package name="doc-count">
                 <filter>IPADDR != '0.0.0.0'</filter>
                 <include-range begin="1.1.1.1" end="254.254.254.254"/>

                 <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
                        <parameter key="http-collection" value="doc-count-1" />
                        <parameter key="retry" value="1" />
                        <parameter key="timeout" value="2000" />
                 </service>
        </package>

         <collector service="HttpDocCount" class-name="org.opennms.netmgt.collectd.HttpCollector" />
 
 
http-datacollection-config.xml (the entire file)
 
<http-datacollection-config
    xmlns:http-dc="
http://xmlns.opennms.org/xsd/config/http-datacollection"
    xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
http://xmlns.opennms.org/xsd/config/http-datacollection http://www.opennms.org/xsd/config/http-datacollection-config.xsd"
    rrdRepository="/opt/opennms/share/rrd/snmp/" >
  <http-collection name="weather">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:2016</rra>
      <rra>RRA:AVERAGE:0.5:12:1488</rra>
      <rra>RRA:AVERAGE:0.5:288:366</rra>
      <rra>RRA:MAX:0.5:288:366</rra>
      <rra>RRA:MIN:0.5:288:366</rra>
    </rrd>
    <uris>
      <uri name="weather-station">
        <url path="/httpcolltest.html"
             matches="(?s).*?Temperature.*?3366FF">([0-9\.]+).*?Humidity.*?3366FF">([0-9]+)
                     .*?Barometer.*?3366FF">([0-9\.]+).*?Wind\sChill.*?3366FF"><small>([0-9\.]+)
                     .*?Heat\sIndex.*?3366FF"></small>([0-9\.]+).*"
             response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="wsTemperature" match-group="1" type="gauge32"/>
          <attrib alias="wsHumidity"    match-group="2" type="gauge32"/>
          <attrib alias="wsBarometer"   match-group="3" type="gauge32"/>
          <attrib alias="wsWindChill"   match-group="4" type="gauge32"/>
          <attrib alias="wsHeatIndex"   match-group="5" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

<http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/doccount.html"
             matches="(?s).*Document Count:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>
</http-datacollection-config>

 
poller-configuration.xml (just the part I added):
 
    <service name="HttpDocCount" interval="300000" user-defined="true" status="on">
      <parameter key="retry" value="3" />
      <parameter key="timeout" value="10000" />
      <parameter key="port" value="80" />
      <parameter key="url" value="/" />
      <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
      <parameter key="rrd-base-name" value="httpdoccount" />
      <parameter key="ds-name" value="httpdoccount" />
    </service>

 
 
 <monitor service="HttpDocCount" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor" />
 
 
The HttpDocCount service has been discovered on that node (192.168.1.171) and is being monitored properly. The httpdoccount.jrb has been showed up inside /opt/opennms/share/rrd/response/192.168.1.171 direcory but nothing has ever appeared in /opt/opennms/share/rrd/snmp/192.168.1.171 directory which means no data has been ever collected on this node.
 
The project deadline is approaching and I still could't get any of the examples to work. I am sure since you have already worked on more explicit projects, this should be piece of cake for you. Would that be possible for you to help me figuring this out?
 
Thanks in advance
 



Date: Wed, 16 Jul 2008 01:56:36 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I have just updated the page (as in 10 minutes ago) that Pete linked, it contains copy and paste code.

The reason the "page" you created does not work is that mod_status generates a dynamic page so if you have it running /server-status will work under apache.

Starting with the server-status page would be the easiest place to begin as the regex is very simplistic, the weather page is a complex pattern match and you would have to dig through log files to see what its really saying.

-Oka
 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Need to know now? Get instant answers with Windows Live Messenger. IM on your terms.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Use video conversation to talk face-to-face with Windows Live Messenger. Get started.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


With Windows Live for mobile, your contacts travel with you. Connect on the go.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Time for vacation? WIN what you need. Enter Now!-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Time for vacation? WIN what you need. Enter Now!
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Data Collector example doesn't work

Byron Anderson

The folder name is the NodeID of the node that the service is on.

 

Response time data is interface specific and therefore is stored based on IP, node specific data is stored based on NodeID.

 

You need to understand that the system that collects the data is completely separate from the system that displays the data hence both need to be setup and configured separately.  To be able to have a graph for this data you will need to manually setup a graph in snmp-graph.properties file.  This can be tough if you are not at least semi-familiar with RRD. 

 

 

 

-------------------------------------------

Byron Anderson

Network Operations Center Manager

EasyStreet Online Services

www.easystreet.com

 


From: [hidden email] [mailto:[hidden email]] On Behalf Of Honia A
Sent: Wednesday, July 16, 2008 11:14 AM
To: General OpenNMS Discussion
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

 

Good news! The data is being collected now! Here are the list of the problems which I was able to find and solve:
 
1) The xml links were all incorrect...so I replaced them with the one I found on ONMS website.
 
2) Removed the HttpWeatherStation service and all related blocks of code from all the files. The value "small" was causing the data collection process to crash.
 
3) Found duplicate files: collectd-configuration.xml, collectd-configuration.xml~ The second one was empty (just the default info), so I copied/pasted the lines from file 1 to file 2. So until I figure out which one is the real one, I will keep both.
 
Now, inside /opt/opennms/share/rrd/snmp/ directory, a new folder was created called "23" which has the following files in it:
 
BusyWorkers.jrb, BytesPerSec.jrb, documentCount.jrb and IdleWorkers.jrb.
 
Now I was wondering if someone could tell me:
 
1 - How to see the content of these files (I understand it's a jrobin file and I probably have the jar file on my system)
 
2 - Why the folder is called 23?
 
3 - Shouldn't this new folder be created inside the /opt/opennms/share/rrd/snmp/192.168.1.171 directory? (I thuoght I added these service to this IP)...
 
Thanks again,
Honia

 
 


 

 

 


Date: Thu, 17 Jul 2008 00:24:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Which version of NMS are you using?

-Oka

Honia A <[hidden email]> wrote:

Oka,
 
This block of code already exist in my log4j.properties and looks like it's been turned on:
 
# Collectd
log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log
log4j.appender.COLLECTD.layout=org.apache.log4j.PatternLayout
log4j.appender.COLLECTD.layout.ConversionPattern=%d %-5p [%t] %c{1}: %m%n
 
 
About the rescan, I had changed the default value for rescan-frequency in capsd-configuration....it's now 10. I did that so that opennms discover the service immediately...
 
Please advise.
 
Thanks


 

 


Date: Wed, 16 Jul 2008 22:07:56 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Go into the etc directory and locate the file log4j.properties, and turn on DEBUG for the collectd.log. You shouldn't need to restart NMS for this, force a rescan on the node and then examine the collectd log.

log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log



Honia A <[hidden email]> wrote:

Oka,
 
I looked at the entire log file (collectd.log) but the word "HttpCollector" hasn't been mentioned anywhere.... No Java block either.
 
I use ssh client (Putty) software to connect to the that server which is located on another machine.
 
Please advise.
 
Thanks,
honia
 


 

 

 


Date: Wed, 16 Jul 2008 05:12:02 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

You will need to paste parts of your collectd.log file search for entries that have this

HttpCollector:

If there is an obvious error you will see a java stack trace associated with it.  Also are you running on the same machine or through a reverse proxy?

This is an example of an error that was generated when there was a misconfigured vhost

XXXX-XX-XX 14:59:52,315 WARN  [CollectdScheduler-50 Pool-fiber1] HttpCollector: doCollection: no attributes defined for collection were found in response t
ext matching regular expression '(?s).*?BusyWorkers:\s([0-9]+).*'
XXXX-XX-XX 14:59:52,315 ERROR [CollectdScheduler-50 Pool-fiber1] HttpCollector: collect: http collection failed: org.opennms.netmgt.collectd.HttpCollector$
HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
org.opennms.netmgt.collectd.HttpCollector$HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
        at org.opennms.netmgt.collectd.HttpCollector.persistResponse(HttpCollector.java:400)
        at org.opennms.netmgt.collectd.HttpCollector.doCollection(HttpCollector.java:253)
        at org.opennms.netmgt.collectd.HttpCollector.access$000(HttpCollector.java:95)
        at org.opennms.netmgt.collectd.HttpCollector$HttpCollectionSet.collect(HttpCollector.java:171)
        at org.opennms.netmgt.collectd.HttpCollector.collect(HttpCollector.java:130)
        at org.opennms.netmgt.collectd.CollectionSpecification.collect(CollectionSpecification.java:223)
        at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:347)
        at org.opennms.netmgt.collectd.CollectableService.run(CollectableService.java:296)
        at org.opennms.netmgt.scheduler.LegacyScheduler$1.run(LegacyScheduler.java:292)

-Oka


Honia A <[hidden email]> wrote:

Hi Oka,
 
Thanks for your reply. I replaced all my code with yours but unfortunately it still doesn't collect the data...
 
On the side note, I used mod_status and got the /server-status to work and the http://192.168.1.171/server-status shows all the data. Then followed the instruction on http://www.opennms.org/index.php/Monitoring_Apache_with_the_HTTP_collector and edited all the config files but still no data is being collected....

Please help :)
 
Thanks

 

 


Date: Wed, 16 Jul 2008 03:05:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I went and updated the Opennms FAQ page with the changes here so this documented.

-Oka

[hidden email] wrote:

I just added the service "with some minor modifications to not disrupt my system" this works and the .jrb file was created.

My capsd-configuration.xml

     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <property key="url" value="/doccount.html" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

collectd-configuration.xml

    <package name="doc-count">
        <filter>IPADDR != '0.0.0.0'</filter>
        <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
            <parameter key="http-collection" value="doc-count-1" />
            <parameter key="retry" value="1" />
            <parameter key="timeout" value="2000" />
            <parameter key="url" value="/doccount.html"/>
         </service>
     </package>

My http-datacollection-config.xml
  <http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/index.html"
             matches="(?s).*?Document\sCount:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

-Oka

Honia A <[hidden email]> wrote:

Hi Oks,
 
Thanks for your reply. I don't have mod_status running and not sure how to get that to work...I have to do a little research on that first. But for now, as I mentioned before, I am also working on the HttpDocCount example and as you may already know, the regex for this example is pretty simple and the data source is almost one line but the data is not being collected... Can you please do me a favor and take a look at the config files below and see what the problem is?
 
doccount.html (located in /var/www/html direcoty):
 

   <html>
     <head>
       <meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
       <meta name="GENERATOR" content="Microsoft FrontPage 4.0">
       <meta name="ProgId" content="FrontPage.Editor.Document">
       <title>collector test</title>
     </head>
     <body>
       <p>Document Count: 5</p>
     </body>
   </html>

 
capsd-configuration.xml (just the part I added):
 
     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

 
collectd-configuration.xml (just the part I added):
 
  <package name="doc-count">
                 <filter>IPADDR != '0.0.0.0'</filter>
                 <include-range begin="1.1.1.1" end="254.254.254.254"/>

                 <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
                        <parameter key="http-collection" value="doc-count-1" />
                        <parameter key="retry" value="1" />
                        <parameter key="timeout" value="2000" />
                 </service>
        </package>

         <collector service="HttpDocCount" class-name="org.opennms.netmgt.collectd.HttpCollector" />
 
 
http-datacollection-config.xml (the entire file)
 
<http-datacollection-config
    xmlns:http-dc="
http://xmlns.opennms.org/xsd/config/http-datacollection"
    xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
http://xmlns.opennms.org/xsd/config/http-datacollection http://www.opennms.org/xsd/config/http-datacollection-config.xsd"
    rrdRepository="/opt/opennms/share/rrd/snmp/" >
  <http-collection name="weather">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:2016</rra>
      <rra>RRA:AVERAGE:0.5:12:1488</rra>
      <rra>RRA:AVERAGE:0.5:288:366</rra>
      <rra>RRA:MAX:0.5:288:366</rra>
      <rra>RRA:MIN:0.5:288:366</rra>
    </rrd>
    <uris>
      <uri name="weather-station">
        <url path="/httpcolltest.html"
             matches="(?s).*?Temperature.*?3366FF">([0-9\.]+).*?Humidity.*?3366FF">([0-9]+)
                     .*?Barometer.*?3366FF">([0-9\.]+).*?Wind\sChill.*?3366FF"><small>([0-9\.]+)
                     .*?Heat\sIndex.*?3366FF"></small>([0-9\.]+).*"
             response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="wsTemperature" match-group="1" type="gauge32"/>
          <attrib alias="wsHumidity"    match-group="2" type="gauge32"/>
          <attrib alias="wsBarometer"   match-group="3" type="gauge32"/>
          <attrib alias="wsWindChill"   match-group="4" type="gauge32"/>
          <attrib alias="wsHeatIndex"   match-group="5" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

<http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/doccount.html"
             matches="(?s).*Document Count:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>
</http-datacollection-config>

 
poller-configuration.xml (just the part I added):
 
    <service name="HttpDocCount" interval="300000" user-defined="true" status="on">
      <parameter key="retry" value="3" />
      <parameter key="timeout" value="10000" />
      <parameter key="port" value="80" />
      <parameter key="url" value="/" />
      <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
      <parameter key="rrd-base-name" value="httpdoccount" />
      <parameter key="ds-name" value="httpdoccount" />
    </service>

 
 
 <monitor service="HttpDocCount" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor" />
 
 
The HttpDocCount service has been discovered on that node (192.168.1.171) and is being monitored properly. The httpdoccount.jrb has been showed up inside /opt/opennms/share/rrd/response/192.168.1.171 direcory but nothing has ever appeared in /opt/opennms/share/rrd/snmp/192.168.1.171 directory which means no data has been ever collected on this node.
 
The project deadline is approaching and I still could't get any of the examples to work. I am sure since you have already worked on more explicit projects, this should be piece of cake for you. Would that be possible for you to help me figuring this out?
 
Thanks in advance

 

 


Date: Wed, 16 Jul 2008 01:56:36 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I have just updated the page (as in 10 minutes ago) that Pete linked, it contains copy and paste code.

The reason the "page" you created does not work is that mod_status generates a dynamic page so if you have it running /server-status will work under apache.

Starting with the server-status page would be the easiest place to begin as the regex is very simplistic, the weather page is a complex pattern match and you would have to dig through log files to see what its really saying.

-Oka

 


Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Need to know now? Get instant answers with Windows Live Messenger. IM on your terms.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

 


Stop! Global Warming ~ Yahoo! JAPAN Earth Project
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

 


Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Use video conversation to talk face-to-face with Windows Live Messenger. Get started.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

 


Stop! Global Warming ~ Yahoo! JAPAN Earth Project


With Windows Live for mobile, your contacts travel with you. Connect on the go.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

 


Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Time for vacation? WIN what you need. Enter Now!-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

 


Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Time for vacation? WIN what you need. Enter Now!


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Start up java error

Jeff Gehlbach
In reply to this post by Mudasar
You hijacked a thread by replying to a message in the thread about the  
HTTP Data Collector example.  Please compose a brand new message to  
the list address when you want to begin discussing a new topic.

-jeff

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Data Collector example doesn't work

tezarin
In reply to this post by Byron Anderson
Thanks for your reply Byron,
 
So the only way to view the collected data will be via using graphs?
 
Thanks again,
honia
 


 
 



Date: Wed, 16 Jul 2008 11:20:54 -0700
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

The folder name is the NodeID of the node that the service is on.

 

Response time data is interface specific and therefore is stored based on IP, node specific data is stored based on NodeID.

 

You need to understand that the system that collects the data is completely separate from the system that displays the data hence both need to be setup and configured separately.  To be able to have a graph for this data you will need to manually setup a graph in snmp-graph.properties file.  This can be tough if you are not at least semi-familiar with RRD. 

 

 

 

-------------------------------------------

Byron Anderson

Network Operations Center Manager

EasyStreet Online Services

www.easystreet.com

 


From: [hidden email] [mailto:[hidden email]] On Behalf Of Honia A
Sent: Wednesday, July 16, 2008 11:14 AM
To: General OpenNMS Discussion
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

 

Good news! The data is being collected now! Here are the list of the problems which I was able to find and solve:
 
1) The xml links were all incorrect...so I replaced them with the one I found on ONMS website.
 
2) Removed the HttpWeatherStation service and all related blocks of code from all the files. The value "small" was causing the data collection process to crash.
 
3) Found duplicate files: collectd-configuration.xml, collectd-configuration.xml~ The second one was empty (just the default info), so I copied/pasted the lines from file 1 to file 2. So until I figure out which one is the real one, I will keep both.
 
Now, inside /opt/opennms/share/rrd/snmp/ directory, a new folder was created called "23" which has the following files in it:
 
BusyWorkers.jrb, BytesPerSec.jrb, documentCount.jrb and IdleWorkers.jrb.
 
Now I was wondering if someone could tell me:
 
1 - How to see the content of these files (I understand it's a jrobin file and I probably have the jar file on my system)
 
2 - Why the folder is called 23?
 
3 - Shouldn't this new folder be created inside the /opt/opennms/share/rrd/snmp/192.168.1.171 directory? (I thuoght I added these service to this IP)...
 
Thanks again,
Honia

 
 


 

 

 


Date: Thu, 17 Jul 2008 00:24:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Which version of NMS are you using?

-Oka

Honia A <[hidden email]> wrote:

Oka,
 
This block of code already exist in my log4j.properties and looks like it's been turned on:
 
# Collectd
log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log
log4j.appender.COLLECTD.layout=org.apache.log4j.PatternLayout
log4j.appender.COLLECTD.layout.ConversionPattern=%d %-5p [%t] %c{1}: %m%n
 
 
About the rescan, I had changed the default value for rescan-frequency in capsd-configuration....it's now 10. I did that so that opennms discover the service immediately...
 
Please advise.
 
Thanks


 

 


Date: Wed, 16 Jul 2008 22:07:56 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Go into the etc directory and locate the file log4j.properties, and turn on DEBUG for the collectd.log. You shouldn't need to restart NMS for this, force a rescan on the node and then examine the collectd log.

log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log



Honia A <[hidden email]> wrote:

Oka,
 
I looked at the entire log file (collectd.log) but the word "HttpCollector" hasn't been mentioned anywhere.... No Java block either.
 
I use ssh client (Putty) software to connect to the that server which is located on another machine.
 
Please advise.
 
Thanks,
honia
 


 

 

 


Date: Wed, 16 Jul 2008 05:12:02 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

You will need to paste parts of your collectd.log file search for entries that have this

HttpCollector:

If there is an obvious error you will see a java stack trace associated with it.  Also are you running on the same machine or through a reverse proxy?

This is an example of an error that was generated when there was a misconfigured vhost

XXXX-XX-XX 14:59:52,315 WARN  [CollectdScheduler-50 Pool-fiber1] HttpCollector: doCollection: no attributes defined for collection were found in response t
ext matching regular expression '(?s).*?BusyWorkers:\s([0-9]+).*'
XXXX-XX-XX 14:59:52,315 ERROR [CollectdScheduler-50 Pool-fiber1] HttpCollector: collect: http collection failed: org.opennms.netmgt.collectd.HttpCollector$
HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
org.opennms.netmgt.collectd.HttpCollector$HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
        at org.opennms.netmgt.collectd.HttpCollector.persistResponse(HttpCollector.java:400)
        at org.opennms.netmgt.collectd.HttpCollector.doCollection(HttpCollector.java:253)
        at org.opennms.netmgt.collectd.HttpCollector.access$000(HttpCollector.java:95)
        at org.opennms.netmgt.collectd.HttpCollector$HttpCollectionSet.collect(HttpCollector.java:171)
        at org.opennms.netmgt.collectd.HttpCollector.collect(HttpCollector.java:130)
        at org.opennms.netmgt.collectd.CollectionSpecification.collect(CollectionSpecification.java:223)
        at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:347)
        at org.opennms.netmgt.collectd.CollectableService.run(CollectableService.java:296)
        at org.opennms.netmgt.scheduler.LegacyScheduler$1.run(LegacyScheduler.java:292)

-Oka


Honia A <[hidden email]> wrote:

Hi Oka,
 
Thanks for your reply. I replaced all my code with yours but unfortunately it still doesn't collect the data...
 
On the side note, I used mod_status and got the /server-status to work and the http://192.168.1.171/server-status shows all the data. Then followed the instruction on http://www.opennms.org/index.php/Monitoring_Apache_with_the_HTTP_collector and edited all the config files but still no data is being collected....

Please help :)
 
Thanks

 

 


Date: Wed, 16 Jul 2008 03:05:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I went and updated the Opennms FAQ page with the changes here so this documented.

-Oka

[hidden email] wrote:

I just added the service "with some minor modifications to not disrupt my system" this works and the .jrb file was created.

My capsd-configuration.xml

     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <property key="url" value="/doccount.html" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

collectd-configuration.xml

    <package name="doc-count">
        <filter>IPADDR != '0.0.0.0'</filter>
        <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
            <parameter key="http-collection" value="doc-count-1" />
            <parameter key="retry" value="1" />
            <parameter key="timeout" value="2000" />
            <parameter key="url" value="/doccount.html"/>
         </service>
     </package>

My http-datacollection-config.xml
  <http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/index.html"
             matches="(?s).*?Document\sCount:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

-Oka

Honia A <[hidden email]> wrote:

Hi Oks,
 
Thanks for your reply. I don't have mod_status running and not sure how to get that to work...I have to do a little research on that first. But for now, as I mentioned before, I am also working on the HttpDocCount example and as you may already know, the regex for this example is pretty simple and the data source is almost one line but the data is not being collected... Can you please do me a favor and take a look at the config files below and see what the problem is?
 
doccount.html (located in /var/www/html direcoty):
 

   <html>
     <head>
       <meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
       <meta name="GENERATOR" content="Microsoft FrontPage 4.0">
       <meta name="ProgId" content="FrontPage.Editor.Document">
       <title>collector test</title>
     </head>
     <body>
       <p>Document Count: 5</p>
     </body>
   </html>

 
capsd-configuration.xml (just the part I added):
 
     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

 
collectd-configuration.xml (just the part I added):
 
  <package name="doc-count">
                 <filter>IPADDR != '0.0.0.0'</filter>
                 <include-range begin="1.1.1.1" end="254.254.254.254"/>

                 <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
                        <parameter key="http-collection" value="doc-count-1" />
                        <parameter key="retry" value="1" />
                        <parameter key="timeout" value="2000" />
                 </service>
        </package>

         <collector service="HttpDocCount" class-name="org.opennms.netmgt.collectd.HttpCollector" />
 
 
http-datacollection-config.xml (the entire file)
 
<http-datacollection-config
    xmlns:http-dc="
http://xmlns.opennms.org/xsd/config/http-datacollection"
    xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
http://xmlns.opennms.org/xsd/config/http-datacollection http://www.opennms.org/xsd/config/http-datacollection-config.xsd"
    rrdRepository="/opt/opennms/share/rrd/snmp/" >
  <http-collection name="weather">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:2016</rra>
      <rra>RRA:AVERAGE:0.5:12:1488</rra>
      <rra>RRA:AVERAGE:0.5:288:366</rra>
      <rra>RRA:MAX:0.5:288:366</rra>
      <rra>RRA:MIN:0.5:288:366</rra>
    </rrd>
    <uris>
      <uri name="weather-station">
        <url path="/httpcolltest.html"
             matches="(?s).*?Temperature.*?3366FF">([0-9\.]+).*?Humidity.*?3366FF">([0-9]+)
                     .*?Barometer.*?3366FF">([0-9\.]+).*?Wind\sChill.*?3366FF"><small>([0-9\.]+)
                     .*?Heat\sIndex.*?3366FF"></small>([0-9\.]+).*"
             response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="wsTemperature" match-group="1" type="gauge32"/>
          <attrib alias="wsHumidity"    match-group="2" type="gauge32"/>
          <attrib alias="wsBarometer"   match-group="3" type="gauge32"/>
          <attrib alias="wsWindChill"   match-group="4" type="gauge32"/>
          <attrib alias="wsHeatIndex"   match-group="5" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

<http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/doccount.html"
             matches="(?s).*Document Count:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>
</http-datacollection-config>

 
poller-configuration.xml (just the part I added):
 
    <service name="HttpDocCount" interval="300000" user-defined="true" status="on">
      <parameter key="retry" value="3" />
      <parameter key="timeout" value="10000" />
      <parameter key="port" value="80" />
      <parameter key="url" value="/" />
      <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
      <parameter key="rrd-base-name" value="httpdoccount" />
      <parameter key="ds-name" value="httpdoccount" />
    </service>

 
 
 <monitor service="HttpDocCount" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor" />
 
 
The HttpDocCount service has been discovered on that node (192.168.1.171) and is being monitored properly. The httpdoccount.jrb has been showed up inside /opt/opennms/share/rrd/response/192.168.1.171 direcory but nothing has ever appeared in /opt/opennms/share/rrd/snmp/192.168.1.171 directory which means no data has been ever collected on this node.
 
The project deadline is approaching and I still could't get any of the examples to work. I am sure since you have already worked on more explicit projects, this should be piece of cake for you. Would that be possible for you to help me figuring this out?
 
Thanks in advance

 

 


Date: Wed, 16 Jul 2008 01:56:36 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I have just updated the page (as in 10 minutes ago) that Pete linked, it contains copy and paste code.

The reason the "page" you created does not work is that mod_status generates a dynamic page so if you have it running /server-status will work under apache.

Starting with the server-status page would be the easiest place to begin as the regex is very simplistic, the weather page is a complex pattern match and you would have to dig through log files to see what its really saying.

-Oka

 


Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Need to know now? Get instant answers with Windows Live Messenger. IM on your terms.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

 


Stop! Global Warming ~ Yahoo! JAPAN Earth Project
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

 


Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Use video conversation to talk face-to-face with Windows Live Messenger. Get started.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

 


Stop! Global Warming ~ Yahoo! JAPAN Earth Project


With Windows Live for mobile, your contacts travel with you. Connect on the go.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

 


Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Time for vacation? WIN what you need. Enter Now!-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

 


Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Time for vacation? WIN what you need. Enter Now!



With Windows Live for mobile, your contacts travel with you. Connect on the go.
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Data Collector example doesn't work

okagau007
Honia,

Glad you figured out your problem, we were scratching our head trying to figure out what your problem was and how you managed to get duplicate files.

If you want to just see a quick picture of what your graph will look like you can use "jrobin-inspector" it should be located in the bin directory.

The file snmp-graph.properties is what controls the drawing of the graphs, you can make changes to this real time without the need to restart NMS, just refresh the page that hosts the graphs.

The link that had the example server-status had the modifications I made to it in addition to the full graphs at the very bottom.

-L

Honia A <[hidden email]> wrote:
Thanks for your reply Byron,
 
So the only way to view the collected data will be via using graphs?
 
Thanks again,
honia
 


 
 



Date: Wed, 16 Jul 2008 11:20:54 -0700
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

The folder name is the NodeID of the node that the service is on.
 
Response time data is interface specific and therefore is stored based on IP, node specific data is stored based on NodeID.
 
You need to understand that the system that collects the data is completely separate from the system that displays the data hence both need to be setup and configured separately.  To be able to have a graph for this data you will need to manually setup a graph in snmp-graph.properties file.  This can be tough if you are not at least semi-familiar with RRD. 
 
 
 
-------------------------------------------
Byron Anderson
Network Operations Center Manager
EasyStreet Online Services
 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Honia A
Sent: Wednesday, July 16, 2008 11:14 AM
To: General OpenNMS Discussion
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work
 
Good news! The data is being collected now! Here are the list of the problems which I was able to find and solve:
 
1) The xml links were all incorrect...so I replaced them with the one I found on ONMS website.
 
2) Removed the HttpWeatherStation service and all related blocks of code from all the files. The value "small" was causing the data collection process to crash.
 
3) Found duplicate files: collectd-configuration.xml, collectd-configuration.xml~ The second one was empty (just the default info), so I copied/pasted the lines from file 1 to file 2. So until I figure out which one is the real one, I will keep both.
 
Now, inside /opt/opennms/share/rrd/snmp/ directory, a new folder was created called "23" which has the following files in it:
 
BusyWorkers.jrb, BytesPerSec.jrb, documentCount.jrb and IdleWorkers.jrb.
 
Now I was wondering if someone could tell me:
 
1 - How to see the content of these files (I understand it's a jrobin file and I probably have the jar file on my system)
 
2 - Why the folder is called 23?
 
3 - Shouldn't this new folder be created inside the /opt/opennms/share/rrd/snmp/192.168.1.171 directory? (I thuoght I added these service to this IP)...
 
Thanks again,
Honia

 
 


 
 
 

Date: Thu, 17 Jul 2008 00:24:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Which version of NMS are you using?

-Oka

Honia A <[hidden email]> wrote:
Oka,
 
This block of code already exist in my log4j.properties and looks like it's been turned on:
 
# Collectd
log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log
log4j.appender.COLLECTD.layout=org.apache.log4j.PatternLayout
log4j.appender.COLLECTD.layout.ConversionPattern=%d %-5p [%t] %c{1}: %m%n
 
 
About the rescan, I had changed the default value for rescan-frequency in capsd-configuration....it's now 10. I did that so that opennms discover the service immediately...
 
Please advise.
 
Thanks


 
 

Date: Wed, 16 Jul 2008 22:07:56 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Go into the etc directory and locate the file log4j.properties, and turn on DEBUG for the collectd.log. You shouldn't need to restart NMS for this, force a rescan on the node and then examine the collectd log.

log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log



Honia A <[hidden email]> wrote:
Oka,
 
I looked at the entire log file (collectd.log) but the word "HttpCollector" hasn't been mentioned anywhere.... No Java block either.
 
I use ssh client (Putty) software to connect to the that server which is located on another machine.
 
Please advise.
 
Thanks,
honia
 


 
 
 

Date: Wed, 16 Jul 2008 05:12:02 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

You will need to paste parts of your collectd.log file search for entries that have this

HttpCollector:

If there is an obvious error you will see a java stack trace associated with it.  Also are you running on the same machine or through a reverse proxy?

This is an example of an error that was generated when there was a misconfigured vhost

XXXX-XX-XX 14:59:52,315 WARN  [CollectdScheduler-50 Pool-fiber1] HttpCollector: doCollection: no attributes defined for collection were found in response t
ext matching regular expression '(?s).*?BusyWorkers:\s([0-9]+).*'
XXXX-XX-XX 14:59:52,315 ERROR [CollectdScheduler-50 Pool-fiber1] HttpCollector: collect: http collection failed: org.opennms.netmgt.collectd.HttpCollector$
HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
org.opennms.netmgt.collectd.HttpCollector$HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
        at org.opennms.netmgt.collectd.HttpCollector.persistResponse(HttpCollector.java:400)
        at org.opennms.netmgt.collectd.HttpCollector.doCollection(HttpCollector.java:253)
        at org.opennms.netmgt.collectd.HttpCollector.access$000(HttpCollector.java:95)
        at org.opennms.netmgt.collectd.HttpCollector$HttpCollectionSet.collect(HttpCollector.java:171)
        at org.opennms.netmgt.collectd.HttpCollector.collect(HttpCollector.java:130)
        at org.opennms.netmgt.collectd.CollectionSpecification.collect(CollectionSpecification.java:223)
        at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:347)
        at org.opennms.netmgt.collectd.CollectableService.run(CollectableService.java:296)
        at org.opennms.netmgt.scheduler.LegacyScheduler$1.run(LegacyScheduler.java:292)

-Oka


Honia A <[hidden email]> wrote:
Hi Oka,
 
Thanks for your reply. I replaced all my code with yours but unfortunately it still doesn't collect the data...
 
On the side note, I used mod_status and got the /server-status to work and the http://192.168.1.171/server-status shows all the data. Then followed the instruction on http://www.opennms.org/index.php/Monitoring_Apache_with_the_HTTP_collector and edited all the config files but still no data is being collected....

Please help :)
 
Thanks
 
 

Date: Wed, 16 Jul 2008 03:05:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I went and updated the Opennms FAQ page with the changes here so this documented.

-Oka

[hidden email] wrote:
I just added the service "with some minor modifications to not disrupt my system" this works and the .jrb file was created.

My capsd-configuration.xml

     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <property key="url" value="/doccount.html" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

collectd-configuration.xml

    <package name="doc-count">
        <filter>IPADDR != '0.0.0.0'</filter>
        <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
            <parameter key="http-collection" value="doc-count-1" />
            <parameter key="retry" value="1" />
            <parameter key="timeout" value="2000" />
            <parameter key="url" value="/doccount.html"/>
         </service>
     </package>

My http-datacollection-config.xml
  <http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/index.html"
             matches="(?s).*?Document\sCount:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

-Oka

Honia A <[hidden email]> wrote:
Hi Oks,
 
Thanks for your reply. I don't have mod_status running and not sure how to get that to work...I have to do a little research on that first. But for now, as I mentioned before, I am also working on the HttpDocCount example and as you may already know, the regex for this example is pretty simple and the data source is almost one line but the data is not being collected... Can you please do me a favor and take a look at the config files below and see what the problem is?
 
doccount.html (located in /var/www/html direcoty):
 

   <html>
     <head>
       <meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
       <meta name="GENERATOR" content="Microsoft FrontPage 4.0">
       <meta name="ProgId" content="FrontPage.Editor.Document">
       <title>collector test</title>
     </head>
     <body>
       <p>Document Count: 5</p>
     </body>
   </html>

 
capsd-configuration.xml (just the part I added):
 
     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

 
collectd-configuration.xml (just the part I added):
 
  <package name="doc-count">
                 <filter>IPADDR != '0.0.0.0'</filter>
                 <include-range begin="1.1.1.1" end="254.254.254.254"/>

                 <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
                        <parameter key="http-collection" value="doc-count-1" />
                        <parameter key="retry" value="1" />
                        <parameter key="timeout" value="2000" />
                 </service>
        </package>

         <collector service="HttpDocCount" class-name="org.opennms.netmgt.collectd.HttpCollector" />
 
 
http-datacollection-config.xml (the entire file)
 
<http-datacollection-config
    xmlns:http-dc="
http://xmlns.opennms.org/xsd/config/http-datacollection"
    xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
http://xmlns.opennms.org/xsd/config/http-datacollection http://www.opennms.org/xsd/config/http-datacollection-config.xsd"
    rrdRepository="/opt/opennms/share/rrd/snmp/" >
  <http-collection name="weather">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:2016</rra>
      <rra>RRA:AVERAGE:0.5:12:1488</rra>
      <rra>RRA:AVERAGE:0.5:288:366</rra>
      <rra>RRA:MAX:0.5:288:366</rra>
      <rra>RRA:MIN:0.5:288:366</rra>
    </rrd>
    <uris>
      <uri name="weather-station">
        <url path="/httpcolltest.html"
             matches="(?s).*?Temperature.*?3366FF">([0-9\.]+).*?Humidity.*?3366FF">([0-9]+)
                     .*?Barometer.*?3366FF">([0-9\.]+).*?Wind\sChill.*?3366FF"><small>([0-9\.]+)
                     .*?Heat\sIndex.*?3366FF"></small>([0-9\.]+).*"
             response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="wsTemperature" match-group="1" type="gauge32"/>
          <attrib alias="wsHumidity"    match-group="2" type="gauge32"/>
          <attrib alias="wsBarometer"   match-group="3" type="gauge32"/>
          <attrib alias="wsWindChill"   match-group="4" type="gauge32"/>
          <attrib alias="wsHeatIndex"   match-group="5" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

<http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/doccount.html"
             matches="(?s).*Document Count:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>
</http-datacollection-config>

 
poller-configuration.xml (just the part I added):
 
    <service name="HttpDocCount" interval="300000" user-defined="true" status="on">
      <parameter key="retry" value="3" />
      <parameter key="timeout" value="10000" />
      <parameter key="port" value="80" />
      <parameter key="url" value="/" />
      <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
      <parameter key="rrd-base-name" value="httpdoccount" />
      <parameter key="ds-name" value="httpdoccount" />
    </service>

 
 
 <monitor service="HttpDocCount" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor" />
 
 
The HttpDocCount service has been discovered on that node (192.168.1.171) and is being monitored properly. The httpdoccount.jrb has been showed up inside /opt/opennms/share/rrd/response/192.168.1.171 direcory but nothing has ever appeared in /opt/opennms/share/rrd/snmp/192.168.1.171 directory which means no data has been ever collected on this node.
 
The project deadline is approaching and I still could't get any of the examples to work. I am sure since you have already worked on more explicit projects, this should be piece of cake for you. Would that be possible for you to help me figuring this out?
 
Thanks in advance
 
 

Date: Wed, 16 Jul 2008 01:56:36 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I have just updated the page (as in 10 minutes ago) that Pete linked, it contains copy and paste code.

The reason the "page" you created does not work is that mod_status generates a dynamic page so if you have it running /server-status will work under apache.

Starting with the server-status page would be the easiest place to begin as the regex is very simplistic, the weather page is a complex pattern match and you would have to dig through log files to see what its really saying.

-Oka
 


Need to know now? Get instant answers with Windows Live Messenger. IM on your terms.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
 
 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
 
 


Use video conversation to talk face-to-face with Windows Live Messenger. Get started.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
 
 


With Windows Live for mobile, your contacts travel with you. Connect on the go.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
 
 


Time for vacation? WIN what you need. Enter Now!-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
 
 


Time for vacation? WIN what you need. Enter Now!


With Windows Live for mobile, your contacts travel with you. Connect on the go.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 


Stop! Global Warming ~ Yahoo! JAPAN Earth Project

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Data Collector example doesn't work

tezarin
Thanks for all your help, Oka.
 
You and Jeff and many other members were nice enough to take time and help me with this "unusual" problem!
jrobin? I found some information on this while ago: http://www.opennms.org/index.php/View_collected_data
 
I would try to use that and see what happens. I really like the changes you made to the Apache-Stats example. Great work and very helpful information :)
 
Thanks,
h
 


 
 



Date: Thu, 17 Jul 2008 22:27:28 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Glad you figured out your problem, we were scratching our head trying to figure out what your problem was and how you managed to get duplicate files.

If you want to just see a quick picture of what your graph will look like you can use "jrobin-inspector" it should be located in the bin directory.

The file snmp-graph.properties is what controls the drawing of the graphs, you can make changes to this real time without the need to restart NMS, just refresh the page that hosts the graphs.

The link that had the example server-status had the modifications I made to it in addition to the full graphs at the very bottom.

-L

Honia A <[hidden email]> wrote:
Thanks for your reply Byron,
 
So the only way to view the collected data will be via using graphs?
 
Thanks again,
honia
 


 
 



Date: Wed, 16 Jul 2008 11:20:54 -0700
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

The folder name is the NodeID of the node that the service is on.
 
Response time data is interface specific and therefore is stored based on IP, node specific data is stored based on NodeID.
 
You need to understand that the system that collects the data is completely separate from the system that displays the data hence both need to be setup and configured separately.  To be able to have a graph for this data you will need to manually setup a graph in snmp-graph.properties file.  This can be tough if you are not at least semi-familiar with RRD. 
 
 
 
-------------------------------------------
Byron Anderson
Network Operations Center Manager
EasyStreet Online Services
 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Honia A
Sent: Wednesday, July 16, 2008 11:14 AM
To: General OpenNMS Discussion
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work
 
Good news! The data is being collected now! Here are the list of the problems which I was able to find and solve:
 
1) The xml links were all incorrect...so I replaced them with the one I found on ONMS website.
 
2) Removed the HttpWeatherStation service and all related blocks of code from all the files. The value "small" was causing the data collection process to crash.
 
3) Found duplicate files: collectd-configuration.xml, collectd-configuration.xml~ The second one was empty (just the default info), so I copied/pasted the lines from file 1 to file 2. So until I figure out which one is the real one, I will keep both.
 
Now, inside /opt/opennms/share/rrd/snmp/ directory, a new folder was created called "23" which has the following files in it:
 
BusyWorkers.jrb, BytesPerSec.jrb, documentCount.jrb and IdleWorkers.jrb.
 
Now I was wondering if someone could tell me:
 
1 - How to see the content of these files (I understand it's a jrobin file and I probably have the jar file on my system)
 
2 - Why the folder is called 23?
 
3 - Shouldn't this new folder be created inside the /opt/opennms/share/rrd/snmp/192.168.1.171 directory? (I thuoght I added these service to this IP)...
 
Thanks again,
Honia

 
 


 
 
 

Date: Thu, 17 Jul 2008 00:24:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Which version of NMS are you using?

-Oka

Honia A <[hidden email]> wrote:
Oka,
 
This block of code already exist in my log4j.properties and looks like it's been turned on:
 
# Collectd
log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log
log4j.appender.COLLECTD.layout=org.apache.log4j.PatternLayout
log4j.appender.COLLECTD.layout.ConversionPattern=%d %-5p [%t] %c{1}: %m%n
 
 
About the rescan, I had changed the default value for rescan-frequency in capsd-configuration....it's now 10. I did that so that opennms discover the service immediately...
 
Please advise.
 
Thanks


 
 

Date: Wed, 16 Jul 2008 22:07:56 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Go into the etc directory and locate the file log4j.properties, and turn on DEBUG for the collectd.log. You shouldn't need to restart NMS for this, force a rescan on the node and then examine the collectd log.

log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log



Honia A <[hidden email]> wrote:
Oka,
 
I looked at the entire log file (collectd.log) but the word "HttpCollector" hasn't been mentioned anywhere.... No Java block either.
 
I use ssh client (Putty) software to connect to the that server which is located on another machine.
 
Please advise.
 
Thanks,
honia
 


 
 
 

Date: Wed, 16 Jul 2008 05:12:02 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

You will need to paste parts of your collectd.log file search for entries that have this

HttpCollector:

If there is an obvious error you will see a java stack trace associated with it.  Also are you running on the same machine or through a reverse proxy?

This is an example of an error that was generated when there was a misconfigured vhost

XXXX-XX-XX 14:59:52,315 WARN  [CollectdScheduler-50 Pool-fiber1] HttpCollector: doCollection: no attributes defined for collection were found in response t
ext matching regular expression '(?s).*?BusyWorkers:\s([0-9]+).*'
XXXX-XX-XX 14:59:52,315 ERROR [CollectdScheduler-50 Pool-fiber1] HttpCollector: collect: http collection failed: org.opennms.netmgt.collectd.HttpCollector$
HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
org.opennms.netmgt.collectd.HttpCollector$HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
        at org.opennms.netmgt.collectd.HttpCollector.persistResponse(HttpCollector.java:400)
        at org.opennms.netmgt.collectd.HttpCollector.doCollection(HttpCollector.java:253)
        at org.opennms.netmgt.collectd.HttpCollector.access$000(HttpCollector.java:95)
        at org.opennms.netmgt.collectd.HttpCollector$HttpCollectionSet.collect(HttpCollector.java:171)
        at org.opennms.netmgt.collectd.HttpCollector.collect(HttpCollector.java:130)
        at org.opennms.netmgt.collectd.CollectionSpecification.collect(CollectionSpecification.java:223)
        at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:347)
        at org.opennms.netmgt.collectd.CollectableService.run(CollectableService.java:296)
        at org.opennms.netmgt.scheduler.LegacyScheduler$1.run(LegacyScheduler.java:292)

-Oka


Honia A <[hidden email]> wrote:
Hi Oka,
 
Thanks for your reply. I replaced all my code with yours but unfortunately it still doesn't collect the data...
 
On the side note, I used mod_status and got the /server-status to work and the http://192.168.1.171/server-status shows all the data. Then followed the instruction on http://www.opennms.org/index.php/Monitoring_Apache_with_the_HTTP_collector and edited all the config files but still no data is being collected....

Please help :)
 
Thanks
 
 

Date: Wed, 16 Jul 2008 03:05:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I went and updated the Opennms FAQ page with the changes here so this documented.

-Oka

[hidden email] wrote:
I just added the service "with some minor modifications to not disrupt my system" this works and the .jrb file was created.

My capsd-configuration.xml

     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <property key="url" value="/doccount.html" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

collectd-configuration.xml

    <package name="doc-count">
        <filter>IPADDR != '0.0.0.0'</filter>
        <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
            <parameter key="http-collection" value="doc-count-1" />
            <parameter key="retry" value="1" />
            <parameter key="timeout" value="2000" />
            <parameter key="url" value="/doccount.html"/>
         </service>
     </package>

My http-datacollection-config.xml
  <http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/index.html"
             matches="(?s).*?Document\sCount:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

-Oka

Honia A <[hidden email]> wrote:
Hi Oks,
 
Thanks for your reply. I don't have mod_status running and not sure how to get that to work...I have to do a little research on that first. But for now, as I mentioned before, I am also working on the HttpDocCount example and as you may already know, the regex for this example is pretty simple and the data source is almost one line but the data is not being collected... Can you please do me a favor and take a look at the config files below and see what the problem is?
 
doccount.html (located in /var/www/html direcoty):
 

   <html>
     <head>
       <meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
       <meta name="GENERATOR" content="Microsoft FrontPage 4.0">
       <meta name="ProgId" content="FrontPage.Editor.Document">
       <title>collector test</title>
     </head>
     <body>
       <p>Document Count: 5</p>
     </body>
   </html>

 
capsd-configuration.xml (just the part I added):
 
     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

 
collectd-configuration.xml (just the part I added):
 
  <package name="doc-count">
                 <filter>IPADDR != '0.0.0.0'</filter>
                 <include-range begin="1.1.1.1" end="254.254.254.254"/>

                 <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
                        <parameter key="http-collection" value="doc-count-1" />
                        <parameter key="retry" value="1" />
                        <parameter key="timeout" value="2000" />
                 </service>
        </package>

         <collector service="HttpDocCount" class-name="org.opennms.netmgt.collectd.HttpCollector" />
 
 
http-datacollection-config.xml (the entire file)
 
<http-datacollection-config
    xmlns:http-dc="
http://xmlns.opennms.org/xsd/config/http-datacollection"
    xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
http://xmlns.opennms.org/xsd/config/http-datacollection http://www.opennms.org/xsd/config/http-datacollection-config.xsd"
    rrdRepository="/opt/opennms/share/rrd/snmp/" >
  <http-collection name="weather">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:2016</rra>
      <rra>RRA:AVERAGE:0.5:12:1488</rra>
      <rra>RRA:AVERAGE:0.5:288:366</rra>
      <rra>RRA:MAX:0.5:288:366</rra>
      <rra>RRA:MIN:0.5:288:366</rra>
    </rrd>
    <uris>
      <uri name="weather-station">
        <url path="/httpcolltest.html"
             matches="(?s).*?Temperature.*?3366FF">([0-9\.]+).*?Humidity.*?3366FF">([0-9]+)
                     .*?Barometer.*?3366FF">([0-9\.]+).*?Wind\sChill.*?3366FF"><small>([0-9\.]+)
                     .*?Heat\sIndex.*?3366FF"></small>([0-9\.]+).*"
             response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="wsTemperature" match-group="1" type="gauge32"/>
          <attrib alias="wsHumidity"    match-group="2" type="gauge32"/>
          <attrib alias="wsBarometer"   match-group="3" type="gauge32"/>
          <attrib alias="wsWindChill"   match-group="4" type="gauge32"/>
          <attrib alias="wsHeatIndex"   match-group="5" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

<http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/doccount.html"
             matches="(?s).*Document Count:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>
</http-datacollection-config>

 
poller-configuration.xml (just the part I added):
 
    <service name="HttpDocCount" interval="300000" user-defined="true" status="on">
      <parameter key="retry" value="3" />
      <parameter key="timeout" value="10000" />
      <parameter key="port" value="80" />
      <parameter key="url" value="/" />
      <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
      <parameter key="rrd-base-name" value="httpdoccount" />
      <parameter key="ds-name" value="httpdoccount" />
    </service>

 
 
 <monitor service="HttpDocCount" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor" />
 
 
The HttpDocCount service has been discovered on that node (192.168.1.171) and is being monitored properly. The httpdoccount.jrb has been showed up inside /opt/opennms/share/rrd/response/192.168.1.171 direcory but nothing has ever appeared in /opt/opennms/share/rrd/snmp/192.168.1.171 directory which means no data has been ever collected on this node.
 
The project deadline is approaching and I still could't get any of the examples to work. I am sure since you have already worked on more explicit projects, this should be piece of cake for you. Would that be possible for you to help me figuring this out?
 
Thanks in advance
 
 

Date: Wed, 16 Jul 2008 01:56:36 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I have just updated the page (as in 10 minutes ago) that Pete linked, it contains copy and paste code.

The reason the "page" you created does not work is that mod_status generates a dynamic page so if you have it running /server-status will work under apache.

Starting with the server-status page would be the easiest place to begin as the regex is very simplistic, the weather page is a complex pattern match and you would have to dig through log files to see what its really saying.

-Oka
 


Need to know now? Get instant answers with Windows Live Messenger. IM on your terms.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
 
 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
 
 


Use video conversation to talk face-to-face with Windows Live Messenger. Get started.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
 
 


With Windows Live for mobile, your contacts travel with you. Connect on the go.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
 
 


Time for vacation? WIN what you need. Enter Now!-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
 
 


Time for vacation? WIN what you need. Enter Now!


With Windows Live for mobile, your contacts travel with you. Connect on the go.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


Use video conversation to talk face-to-face with Windows Live Messenger. Get started.
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Data Collector example doesn't work

tezarin
In reply to this post by okagau007
Update:
 
Used the information on http://www.opennms.org/index.php/View_collected_data page. But it doesn't work because X11 variable value is not set...
 
This is the command I tried to run: echo -e dump multiicmp.jrb\\n . | java -jar $OPENNMS_HOME/lib/jrobin-1.5.8.jar

This is the error I got:
 
== JRobin's RRDTool commander ==
Type a RRDTool command after the dollar sign and press Enter.
Start your RRDTool command with 'create', 'update', 'fetch' etc.
Start line with 'create', 'update', 'fetch' etc.
Enter dot ('.') to bail out
Current directory is: /opt/opennms/lib
================================
$ java.io.FileNotFoundException: Could not open multiicmp.jrb [non existent]
   at org.jrobin.core.RrdDb.<init>(RrdDb.java:227)
   at org.jrobin.core.RrdDb.<init>(RrdDb.java:203)
   at org.jrobin.core.RrdDb.<init>(RrdDb.java:266)
   at org.jrobin.cmd.RrdToolCmd.getRrdDbReference(RrdToolCmd.java:134)
   at org.jrobin.cmd.RrdDumpCmd.execute(RrdDumpCmd.java:44)
   at org.jrobin.cmd.RrdToolCmd.executeCommand(RrdToolCmd.java:45)
   at org.jrobin.cmd.RrdCommander.execute(RrdCommander.java:134)
   at org.jrobin.cmd.RrdCommander.main(RrdCommander.java:162)

 
Please advise.
 
Thanks


 



Date: Thu, 17 Jul 2008 22:27:28 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Glad you figured out your problem, we were scratching our head trying to figure out what your problem was and how you managed to get duplicate files.

If you want to just see a quick picture of what your graph will look like you can use "jrobin-inspector" it should be located in the bin directory.

The file snmp-graph.properties is what controls the drawing of the graphs, you can make changes to this real time without the need to restart NMS, just refresh the page that hosts the graphs.

The link that had the example server-status had the modifications I made to it in addition to the full graphs at the very bottom.

-L

Honia A <[hidden email]> wrote:
Thanks for your reply Byron,
 
So the only way to view the collected data will be via using graphs?
 
Thanks again,
honia
 


 
 



Date: Wed, 16 Jul 2008 11:20:54 -0700
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

The folder name is the NodeID of the node that the service is on.
 
Response time data is interface specific and therefore is stored based on IP, node specific data is stored based on NodeID.
 
You need to understand that the system that collects the data is completely separate from the system that displays the data hence both need to be setup and configured separately.  To be able to have a graph for this data you will need to manually setup a graph in snmp-graph.properties file.  This can be tough if you are not at least semi-familiar with RRD. 
 
 
 
-------------------------------------------
Byron Anderson
Network Operations Center Manager
EasyStreet Online Services
 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Honia A
Sent: Wednesday, July 16, 2008 11:14 AM
To: General OpenNMS Discussion
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work
 
Good news! The data is being collected now! Here are the list of the problems which I was able to find and solve:
 
1) The xml links were all incorrect...so I replaced them with the one I found on ONMS website.
 
2) Removed the HttpWeatherStation service and all related blocks of code from all the files. The value "small" was causing the data collection process to crash.
 
3) Found duplicate files: collectd-configuration.xml, collectd-configuration.xml~ The second one was empty (just the default info), so I copied/pasted the lines from file 1 to file 2. So until I figure out which one is the real one, I will keep both.
 
Now, inside /opt/opennms/share/rrd/snmp/ directory, a new folder was created called "23" which has the following files in it:
 
BusyWorkers.jrb, BytesPerSec.jrb, documentCount.jrb and IdleWorkers.jrb.
 
Now I was wondering if someone could tell me:
 
1 - How to see the content of these files (I understand it's a jrobin file and I probably have the jar file on my system)
 
2 - Why the folder is called 23?
 
3 - Shouldn't this new folder be created inside the /opt/opennms/share/rrd/snmp/192.168.1.171 directory? (I thuoght I added these service to this IP)...
 
Thanks again,
Honia

 
 


 
 
 

Date: Thu, 17 Jul 2008 00:24:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Which version of NMS are you using?

-Oka

Honia A <[hidden email]> wrote:
Oka,
 
This block of code already exist in my log4j.properties and looks like it's been turned on:
 
# Collectd
log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log
log4j.appender.COLLECTD.layout=org.apache.log4j.PatternLayout
log4j.appender.COLLECTD.layout.ConversionPattern=%d %-5p [%t] %c{1}: %m%n
 
 
About the rescan, I had changed the default value for rescan-frequency in capsd-configuration....it's now 10. I did that so that opennms discover the service immediately...
 
Please advise.
 
Thanks


 
 

Date: Wed, 16 Jul 2008 22:07:56 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

Honia,

Go into the etc directory and locate the file log4j.properties, and turn on DEBUG for the collectd.log. You shouldn't need to restart NMS for this, force a rescan on the node and then examine the collectd log.

log4j.category.OpenNMS.Collectd=DEBUG, COLLECTD
log4j.additivity.OpenNMS.Collectd=false
log4j.appender.COLLECTD=org.apache.log4j.RollingFileAppender
log4j.appender.COLLECTD.MaxFileSize=100MB
log4j.appender.COLLECTD.MaxBackupIndex=4
log4j.appender.COLLECTD.File=/opt/opennms/logs/daemon/collectd.log



Honia A <[hidden email]> wrote:
Oka,
 
I looked at the entire log file (collectd.log) but the word "HttpCollector" hasn't been mentioned anywhere.... No Java block either.
 
I use ssh client (Putty) software to connect to the that server which is located on another machine.
 
Please advise.
 
Thanks,
honia
 


 
 
 

Date: Wed, 16 Jul 2008 05:12:02 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

You will need to paste parts of your collectd.log file search for entries that have this

HttpCollector:

If there is an obvious error you will see a java stack trace associated with it.  Also are you running on the same machine or through a reverse proxy?

This is an example of an error that was generated when there was a misconfigured vhost

XXXX-XX-XX 14:59:52,315 WARN  [CollectdScheduler-50 Pool-fiber1] HttpCollector: doCollection: no attributes defined for collection were found in response t
ext matching regular expression '(?s).*?BusyWorkers:\s([0-9]+).*'
XXXX-XX-XX 14:59:52,315 ERROR [CollectdScheduler-50 Pool-fiber1] HttpCollector: collect: http collection failed: org.opennms.netmgt.collectd.HttpCollector$
HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
org.opennms.netmgt.collectd.HttpCollector$HttpCollectorException: No attributes specified were found: : client URL: HostConfiguration[]
        at org.opennms.netmgt.collectd.HttpCollector.persistResponse(HttpCollector.java:400)
        at org.opennms.netmgt.collectd.HttpCollector.doCollection(HttpCollector.java:253)
        at org.opennms.netmgt.collectd.HttpCollector.access$000(HttpCollector.java:95)
        at org.opennms.netmgt.collectd.HttpCollector$HttpCollectionSet.collect(HttpCollector.java:171)
        at org.opennms.netmgt.collectd.HttpCollector.collect(HttpCollector.java:130)
        at org.opennms.netmgt.collectd.CollectionSpecification.collect(CollectionSpecification.java:223)
        at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:347)
        at org.opennms.netmgt.collectd.CollectableService.run(CollectableService.java:296)
        at org.opennms.netmgt.scheduler.LegacyScheduler$1.run(LegacyScheduler.java:292)

-Oka


Honia A <[hidden email]> wrote:
Hi Oka,
 
Thanks for your reply. I replaced all my code with yours but unfortunately it still doesn't collect the data...
 
On the side note, I used mod_status and got the /server-status to work and the http://192.168.1.171/server-status shows all the data. Then followed the instruction on http://www.opennms.org/index.php/Monitoring_Apache_with_the_HTTP_collector and edited all the config files but still no data is being collected....

Please help :)
 
Thanks
 
 

Date: Wed, 16 Jul 2008 03:05:59 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I went and updated the Opennms FAQ page with the changes here so this documented.

-Oka

[hidden email] wrote:
I just added the service "with some minor modifications to not disrupt my system" this works and the .jrb file was created.

My capsd-configuration.xml

     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <property key="url" value="/doccount.html" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

collectd-configuration.xml

    <package name="doc-count">
        <filter>IPADDR != '0.0.0.0'</filter>
        <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
            <parameter key="http-collection" value="doc-count-1" />
            <parameter key="retry" value="1" />
            <parameter key="timeout" value="2000" />
            <parameter key="url" value="/doccount.html"/>
         </service>
     </package>

My http-datacollection-config.xml
  <http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/index.html"
             matches="(?s).*?Document\sCount:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

-Oka

Honia A <[hidden email]> wrote:
Hi Oks,
 
Thanks for your reply. I don't have mod_status running and not sure how to get that to work...I have to do a little research on that first. But for now, as I mentioned before, I am also working on the HttpDocCount example and as you may already know, the regex for this example is pretty simple and the data source is almost one line but the data is not being collected... Can you please do me a favor and take a look at the config files below and see what the problem is?
 
doccount.html (located in /var/www/html direcoty):
 

   <html>
     <head>
       <meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
       <meta name="GENERATOR" content="Microsoft FrontPage 4.0">
       <meta name="ProgId" content="FrontPage.Editor.Document">
       <title>collector test</title>
     </head>
     <body>
       <p>Document Count: 5</p>
     </body>
   </html>

 
capsd-configuration.xml (just the part I added):
 
     <protocol-plugin protocol="HttpDocCount" class-name="org.opennms.netmgt.capsd.plugins.HttpPlugin" scan="off" user-defined="true">
        <property key="port" value="80" />
        <property key="timeout" value="3000" />
        <property key="retry" value="1" />
        <protocol-configuration scan="enable" user-defined="true">
            <specific>192.168.1.171</specific>
        </protocol-configuration>
    </protocol-plugin>

 
collectd-configuration.xml (just the part I added):
 
  <package name="doc-count">
                 <filter>IPADDR != '0.0.0.0'</filter>
                 <include-range begin="1.1.1.1" end="254.254.254.254"/>

                 <service name="HttpDocCount" interval="300000" user-defined="false" status="on" >
                        <parameter key="http-collection" value="doc-count-1" />
                        <parameter key="retry" value="1" />
                        <parameter key="timeout" value="2000" />
                 </service>
        </package>

         <collector service="HttpDocCount" class-name="org.opennms.netmgt.collectd.HttpCollector" />
 
 
http-datacollection-config.xml (the entire file)
 
<http-datacollection-config
    xmlns:http-dc="
http://xmlns.opennms.org/xsd/config/http-datacollection"
    xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
http://xmlns.opennms.org/xsd/config/http-datacollection http://www.opennms.org/xsd/config/http-datacollection-config.xsd"
    rrdRepository="/opt/opennms/share/rrd/snmp/" >
  <http-collection name="weather">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:2016</rra>
      <rra>RRA:AVERAGE:0.5:12:1488</rra>
      <rra>RRA:AVERAGE:0.5:288:366</rra>
      <rra>RRA:MAX:0.5:288:366</rra>
      <rra>RRA:MIN:0.5:288:366</rra>
    </rrd>
    <uris>
      <uri name="weather-station">
        <url path="/httpcolltest.html"
             matches="(?s).*?Temperature.*?3366FF">([0-9\.]+).*?Humidity.*?3366FF">([0-9]+)
                     .*?Barometer.*?3366FF">([0-9\.]+).*?Wind\sChill.*?3366FF"><small>([0-9\.]+)
                     .*?Heat\sIndex.*?3366FF"></small>([0-9\.]+).*"
             response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="wsTemperature" match-group="1" type="gauge32"/>
          <attrib alias="wsHumidity"    match-group="2" type="gauge32"/>
          <attrib alias="wsBarometer"   match-group="3" type="gauge32"/>
          <attrib alias="wsWindChill"   match-group="4" type="gauge32"/>
          <attrib alias="wsHeatIndex"   match-group="5" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>

<http-collection name="doc-count-1">
    <rrd step="300">
      <rra>RRA:AVERAGE:0.5:1:8928</rra>
      <rra>RRA:AVERAGE:0.5:12:8784</rra>
      <rra>RRA:MIN:0.5:12:8784</rra>
      <rra>RRA:MAX:0.5:12:8784</rra>
    </rrd>
    <uris>
      <uri name="document-counts">
        <url path="/doccount.html"
             matches="(?s).*Document Count:\s+([0-9]+).*" response-range="100-399" >
        </url>
        <attributes>
          <attrib alias="documentCount" match-group="1" type="gauge32"/>
        </attributes>
      </uri>
    </uris>
  </http-collection>
</http-datacollection-config>

 
poller-configuration.xml (just the part I added):
 
    <service name="HttpDocCount" interval="300000" user-defined="true" status="on">
      <parameter key="retry" value="3" />
      <parameter key="timeout" value="10000" />
      <parameter key="port" value="80" />
      <parameter key="url" value="/" />
      <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
      <parameter key="rrd-base-name" value="httpdoccount" />
      <parameter key="ds-name" value="httpdoccount" />
    </service>

 
 
 <monitor service="HttpDocCount" class-name="org.opennms.netmgt.poller.monitors.HttpMonitor" />
 
 
The HttpDocCount service has been discovered on that node (192.168.1.171) and is being monitored properly. The httpdoccount.jrb has been showed up inside /opt/opennms/share/rrd/response/192.168.1.171 direcory but nothing has ever appeared in /opt/opennms/share/rrd/snmp/192.168.1.171 directory which means no data has been ever collected on this node.
 
The project deadline is approaching and I still could't get any of the examples to work. I am sure since you have already worked on more explicit projects, this should be piece of cake for you. Would that be possible for you to help me figuring this out?
 
Thanks in advance
 
 

Date: Wed, 16 Jul 2008 01:56:36 +0900
From: [hidden email]
To: [hidden email]
Subject: Re: [opennms-discuss] HTTP Data Collector example doesn't work

I have just updated the page (as in 10 minutes ago) that Pete linked, it contains copy and paste code.

The reason the "page" you created does not work is that mod_status generates a dynamic page so if you have it running /server-status will work under apache.

Starting with the server-status page would be the easiest place to begin as the regex is very simplistic, the weather page is a complex pattern match and you would have to dig through log files to see what its really saying.

-Oka
 


Need to know now? Get instant answers with Windows Live Messenger. IM on your terms.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
 
 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
 
 


Use video conversation to talk face-to-face with Windows Live Messenger. Get started.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
 
 


With Windows Live for mobile, your contacts travel with you. Connect on the go.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
 
 


Time for vacation? WIN what you need. Enter Now!-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
 
 


Time for vacation? WIN what you need. Enter Now!


With Windows Live for mobile, your contacts travel with you. Connect on the go.-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

 

Stop! Global Warming ~ Yahoo! JAPAN Earth Project


With Windows Live for mobile, your contacts travel with you. Connect on the go.
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
12