DNS Provisioning causes flood of nodeUpdated events

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

DNS Provisioning causes flood of nodeUpdated events

Jonathan Heard
Hello all,

    I've been experimenting with using DNS as a source for Provisiond as
per https://wiki.opennms.org/wiki/DNS_Importing, which in our
environment would be an excellent way to keep OpenNMS up to date and
it's immediately updated when new VMs are provisioned. I've got it
working, and by using a foreign source and expressions in the DNS URL in
provisiond-configuration.xml it's happily populating my requisitions and
discovering host services - e.g.

<rescan-existing>dbonly</rescan-existing>
   <requisition-def import-name="test"
import-url-resource="dns://ns01/my.zone/test/?expression=^test-.*">
     <cron-schedule>* 5/30 * * * ? *</cron-schedule>
   </requisition-def>
   <requisition-def import-name="admin"
import-url-resource="dns://ns01/my.zone/admin/?expression=^admin-.*">
     <cron-schedule>* 1/30 * * * ? *</cron-schedule>
   </requisition-def>

I want it to scan DNS very regularly (possibly even more than every
30mins a above) but at the moment every new import from DNS causes
approximately 10 uei.opennms.org/nodes/nodeUpdated events for every
host, even though those hosts are already in the requisition and have
not changed :-(

I've tried disabling the following in opennms.properties too:

org.opennms.provisiond.scheduleRescanForExistingNodes=false
org.opennms.provisiond.scheduleRescanForUpdatedNodes=false

 From a look through the DNS provisioning code it looks to me like it
always builds the whole requisition from scratch (i.e. although it uses
an IXFR 'incremental' DNS zone transfer, the SOA serial number is
hardcoded to 0 so it gets the whole zone every time), but the
Hostname/IP and Foreign Source+ID  never changes. (Sidenote: The Foreign
IDs assigned by DNS are just an integer and many of them are a negative
number - Is this correct behaviour?)

What I can't figure out as yet is why so many nodeUpdated events are
triggered for each host (Anyone know if this is normal?), and whether
there's a way and I can force OpenNMS to only treat the node as
'updated' if something about that node has actually changed?

Thanks in advance for any advice.

Regards

Jonathan


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: DNS Provisioning causes flood of nodeUpdated events

Jonathan Heard
I've raised https://issues.opennms.org/browse/NMS-9492 for this problem
and submitted a patch, which cures this for me. This is my first journey
into the OpenNMS code and development practices so I hope it's up to
scratch :-)


On 30/06/17 17:14, Jonathan Heard wrote:

> Hello all,
>
>    I've been experimenting with using DNS as a source for Provisiond
> as per https://wiki.opennms.org/wiki/DNS_Importing, which in our
> environment would be an excellent way to keep OpenNMS up to date and
> it's immediately updated when new VMs are provisioned. I've got it
> working, and by using a foreign source and expressions in the DNS URL
> in provisiond-configuration.xml it's happily populating my
> requisitions and discovering host services - e.g.
>
> <rescan-existing>dbonly</rescan-existing>
>   <requisition-def import-name="test"
> import-url-resource="dns://ns01/my.zone/test/?expression=^test-.*">
>     <cron-schedule>* 5/30 * * * ? *</cron-schedule>
>   </requisition-def>
>   <requisition-def import-name="admin"
> import-url-resource="dns://ns01/my.zone/admin/?expression=^admin-.*">
>     <cron-schedule>* 1/30 * * * ? *</cron-schedule>
>   </requisition-def>
>
> I want it to scan DNS very regularly (possibly even more than every
> 30mins a above) but at the moment every new import from DNS causes
> approximately 10 uei.opennms.org/nodes/nodeUpdated events for every
> host, even though those hosts are already in the requisition and have
> not changed :-(
>
> I've tried disabling the following in opennms.properties too:
>
> org.opennms.provisiond.scheduleRescanForExistingNodes=false
> org.opennms.provisiond.scheduleRescanForUpdatedNodes=false
>
> From a look through the DNS provisioning code it looks to me like it
> always builds the whole requisition from scratch (i.e. although it
> uses an IXFR 'incremental' DNS zone transfer, the SOA serial number is
> hardcoded to 0 so it gets the whole zone every time), but the
> Hostname/IP and Foreign Source+ID  never changes. (Sidenote: The
> Foreign IDs assigned by DNS are just an integer and many of them are a
> negative number - Is this correct behaviour?)
>
> What I can't figure out as yet is why so many nodeUpdated events are
> triggered for each host (Anyone know if this is normal?), and whether
> there's a way and I can force OpenNMS to only treat the node as
> 'updated' if something about that node has actually changed?
>
> Thanks in advance for any advice.
>
> Regards
>
> Jonathan
>
>
> ------------------------------------------------------------------------------
>
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Please read the OpenNMS Mailing List FAQ:
> http://www.opennms.org/index.php/Mailing_List_FAQ
>
> opennms-discuss mailing list
>
> To *unsubscribe* or change your subscription options, see the bottom
> of this page:
> https://lists.sourceforge.net/lists/listinfo/opennms-discuss


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss