Requirement for Parameters defined at the node level for use in collectd-configuration and pollerd-configuration

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Requirement for Parameters defined at the node level for use in collectd-configuration and pollerd-configuration

BeeJay

Due to the number of client databases we are monitoring, we have an ever increasing number of packages in collectd-configuration and pollerd-configuration.

 

The problem is that for every database we need a different jdbc connection string due to differences in port number and service name.

 

The ONLY difference between all these packages is the PORT and SERVICE_NAME tags:

 

<parameter key="url" value="jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=OPENNMS_JDBC_HOSTNAME)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=devdb01)))"/>

 

There is a variable for the HOST tag, but due to having to hard code the port number and service name, we have had to replicate the database monitoring packages for every database.

 

Is it possible to variablize the PORT and SERVICE_NAME parameters (or the whole “URL” parameter key), so that we can reuse the same package for monitoring all our databases (or at least significantly reduce the number of required packages)?

 

Regards,

John Blackburn.

 


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Requirement for Parameters defined at the node level for use in collectd-configuration and pollerd-configuration

David Hustace

On Jun 26, 2017, at 01:12, JohnD Blackburn <[hidden email]> wrote:

Is it possible to variablize the PORT and SERVICE_NAME parameters (or the whole “URL” parameter key), so that we can reuse the same package for monitoring all our databases (or at least significantly reduce the number of required packages)?

John,

This is something that I've wanted to improve in OpenNMS for quite some time.  One way to address it would be to storing the attributes determined from the poller package with the monitored service when they're provisioned.  That way, we don't require complex package configurations, possibly one for every service, to handle variations on the way we want to monitor a resource with any of the protocol monitors.  A simple URL formatted string field stored with the service and a "URL Handler" for each of the ServiceMonitor classes would get us what we need.  DevJam is coming up, perhaps if we add this as an enhancement someone will pick it up or possibly some organization will sponsor it. :)


For example:

SELECT ifs.id, 
       s.servicename, 
       ip.ipaddr, 
       ip.iphostname, 
       ifs.url 
  FROM ifservices ifs 
  JOIN service s 
    ON s.serviceid = ifs.serviceid 
  JOIN ipinterface ip 
    ON ip.id = ifs.ipinterfaceid 
 WHERE ip.id = 9999999;

   id    | servicename |    ipaddr     |  iphostname   |                               url
---------+-------------+---------------+---------------+------------------------------------------------------------------
  353553 | ICMP        | 10.100.58.116 | 10.100.58.116 | <a href="icmp://10.100.58.116?retries=1&amp;timeout=800" class="">icmp://10.100.58.116?retries=1&timeout=800
 1075355 | HTTPS       | 10.100.58.116 | 10.100.58.116 | https://10.100.58.116:443/opennms/alarm/list.htm?display=long
  353555 | SNMP        | 10.100.58.116 | 10.100.58.116 | <a href="snmp://10.100.58.116?retries=3&amp;timeout=800&amp;read-community=public" class="">snmp://10.100.58.116?retries=3&timeout=800&read-community=public


So, in this case we just create a URL handlers for the IcmpMonitor, HttpsMonitor, and SnmpMonitor classes.


Cheers,
David


David Hustace
The OpenNMS Group, Inc.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

signature.asc (210 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Requirement for Parameters defined at the node level for use in collectd-configuration and pollerd-configuration

Ronald Roskens

On Jun 26, 2017, at 12:33 PM, David Hustace <[hidden email]> wrote:


On Jun 26, 2017, at 01:12, JohnD Blackburn <[hidden email]> wrote:

Is it possible to variablize the PORT and SERVICE_NAME parameters (or the whole “URL” parameter key), so that we can reuse the same package for monitoring all our databases (or at least significantly reduce the number of required packages)?

John,

This is something that I've wanted to improve in OpenNMS for quite some time.  One way to address it would be to storing the attributes determined from the poller package with the monitored service when they're provisioned.  That way, we don't require complex package configurations, possibly one for every service, to handle variations on the way we want to monitor a resource with any of the protocol monitors.  A simple URL formatted string field stored with the service and a "URL Handler" for each of the ServiceMonitor classes would get us what we need.  DevJam is coming up, perhaps if we add this as an enhancement someone will pick it up or possibly some organization will sponsor it. :)


For example:

SELECT ifs.id, 
       s.servicename, 
       ip.ipaddr, 
       ip.iphostname, 
       ifs.url 
  FROM ifservices ifs 
  JOIN service s 
    ON s.serviceid = ifs.serviceid 
  JOIN ipinterface ip 
    ON ip.id = ifs.ipinterfaceid 
 WHERE ip.id = 9999999;

   id    | servicename |    ipaddr     |  iphostname   |                               url
---------+-------------+---------------+---------------+------------------------------------------------------------------
  353553 | ICMP        | 10.100.58.116 | 10.100.58.116 | <a href="icmp://10.100.58.116?retries=1&amp;timeout=800" class="">icmp://10.100.58.116?retries=1&timeout=800
 1075355 | HTTPS       | 10.100.58.116 | 10.100.58.116 | https://10.100.58.116:443/opennms/alarm/list.htm?display=long
  353555 | SNMP        | 10.100.58.116 | 10.100.58.116 | <a href="snmp://10.100.58.116?retries=3&amp;timeout=800&amp;read-community=public" class="">snmp://10.100.58.116?retries=3&timeout=800&read-community=public


So, in this case we just create a URL handlers for the IcmpMonitor, HttpsMonitor, and SnmpMonitor classes.

I have to say this would be pretty nice to have. I’m also having to create multiple service definitions each time I spin up a new group of systems just because the port # changes. Having a way to store this at the node / group / category / provisioning requisition would be nice.

I’ve made an entry for this under this years dev-jam page.

Ron

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Requirement for Parameters defined at the node level for use in collectd-configuration and pollerd-configuration

Ronny Trommer-3
Hi there,

I have used a few other monitoring tools before and they had the possibility to set test parameters in a monitor or a data collection from from generic to specific. Translated to OpenNMS it would mean you define a generic behaviour in a package and you are able to set test configuration parameters based on a category membership, node membership and last not least on an IP interface. It would be useful to set node and IP interface specific parameter sets during provisioning.

We started something similar in Google Summer of Code 2013 for Pollerd [1]. There where some issues how Pollerd is designed, Christian and Dustin can explain that more in detail. It would make sense to discuss the conceptional goals for pollerd and collectd in the Wiki [2] to build a shared understanding.


tl;dr

Personally I’m not a friend of misusing URLs and overwriting URL handlers. Every implementation has to build own error prone parser which are hard to validate, with JVM wide side effects and we have a key value pairs in our configs already. They are handled well and can be easily persisted, validated and are easy to understood. We have a similar place where URL handlers are used when you implement a Provisioning adapter We had to use to build the VMware importer. Instead of implementing against APIs where your IDE is telling you what you have to do, you have to build custom URL string parser and you have to do some workarounds which ends up in expensive nasty glue code. Most of the time you need additional configuration parameters beside the URL anyways cause they are limited, e.g. authentication.

On 27. Jun 2017, at 03:49, Ronald Roskens <[hidden email]> wrote:


On Jun 26, 2017, at 12:33 PM, David Hustace <[hidden email]> wrote:


On Jun 26, 2017, at 01:12, JohnD Blackburn <[hidden email]> wrote:

Is it possible to variablize the PORT and SERVICE_NAME parameters (or the whole “URL” parameter key), so that we can reuse the same package for monitoring all our databases (or at least significantly reduce the number of required packages)?

John,

This is something that I've wanted to improve in OpenNMS for quite some time.  One way to address it would be to storing the attributes determined from the poller package with the monitored service when they're provisioned.  That way, we don't require complex package configurations, possibly one for every service, to handle variations on the way we want to monitor a resource with any of the protocol monitors.  A simple URL formatted string field stored with the service and a "URL Handler" for each of the ServiceMonitor classes would get us what we need.  DevJam is coming up, perhaps if we add this as an enhancement someone will pick it up or possibly some organization will sponsor it. :)


For example:

SELECT ifs.id, 
       s.servicename, 
       ip.ipaddr, 
       ip.iphostname, 
       ifs.url 
  FROM ifservices ifs 
  JOIN service s 
    ON s.serviceid = ifs.serviceid 
  JOIN ipinterface ip 
    ON ip.id = ifs.ipinterfaceid 
 WHERE ip.id = 9999999;

   id    | servicename |    ipaddr     |  iphostname   |                               url
---------+-------------+---------------+---------------+------------------------------------------------------------------
  353553 | ICMP        | 10.100.58.116 | 10.100.58.116 | <a href="icmp://10.100.58.116?retries=1&amp;timeout=800" class="">icmp://10.100.58.116?retries=1&timeout=800
 1075355 | HTTPS       | 10.100.58.116 | 10.100.58.116 | https://10.100.58.116:443/opennms/alarm/list.htm?display=long
  353555 | SNMP        | 10.100.58.116 | 10.100.58.116 | <a href="snmp://10.100.58.116?retries=3&amp;timeout=800&amp;read-community=public" class="">snmp://10.100.58.116?retries=3&timeout=800&read-community=public


So, in this case we just create a URL handlers for the IcmpMonitor, HttpsMonitor, and SnmpMonitor classes.

I have to say this would be pretty nice to have. I’m also having to create multiple service definitions each time I spin up a new group of systems just because the port # changes. Having a way to store this at the node / group / category / provisioning requisition would be nice.

I’ve made an entry for this under this years dev-jam page.

Ron
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

signature.asc (507 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Requirement for Parameters defined at the node level for use in collectd-configuration and pollerd-configuration

David Hustace

> On Jun 27, 2017, at 03:15, Ronny Trommer <[hidden email]> wrote:
>
> Personally I’m not a friend of misusing URLs and overwriting URL handlers. Every implementation has to build own error prone parser which are hard to validate, with JVM wide side effects and we have a key value pairs in our configs already.

I know that, Ronny. It's just the easiest way to explain it.  I just didn't want to get into the technical details but we have already improved upon that strategy in Provisiond with the release of H20. I would suggest we would do the same type of thing without having to using the Java official URL handler API.


David Hustace
The OpenNMS Group, Inc.

PS
I knew this was coming from you sooner or later. :)

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

signature.asc (210 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Requirement for Parameters defined at the node level for use in collectd-configuration and pollerd-configuration

Ronny Trommer-3

On 27. Jun 2017, at 22:57, David Hustace <[hidden email]> wrote:


On Jun 27, 2017, at 03:15, Ronny Trommer <[hidden email]> wrote:

Personally I’m not a friend of misusing URLs and overwriting URL handlers. Every implementation has to build own error prone parser which are hard to validate, with JVM wide side effects and we have a key value pairs in our configs already.

I know that, Ronny. It's just the easiest way to explain it.  I just didn't want to get into the technical details but we have already improved upon that strategy in Provisiond with the release of H20. I would suggest we would do the same type of thing without having to using the Java official URL handler API.

If it means you have to configure a connection through something like <a href="vmware://user:pass@vmware-center/foobar?importVMware=true" class="">vmware://user:pass@vmware-center/foobar?importVMware=true using an in-official homebrew URL handler API does not solve the problems I’ve described beside the JVM wide side effect.

Just following the German paradigm something smells fishy to me.

- complain early, complain often :D


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

signature.asc (507 bytes) Download Attachment