Unexpected exception when collecting SNMP data for interface

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Unexpected exception when collecting SNMP data for interface

taraska90
This post was updated on .
Hello firends,

I have issue with "Unexpected exception"

Here log from collectd.log:
cat /var/log/opennms/collectd.log  | grep "Unexpected exception"
org.opennms.netmgt.collectd.CollectionWarning: Unexpected exception when collecting SNMP data for interface 172.16.1.137 at location Default.
2017-03-14 09:18:47,280 WARN  [Collectd-Thread-27-of-50] o.o.n.c.CollectableService: Unexpected exception when collecting SNMP data for interface 172.16.1.134 at location Default.
org.opennms.netmgt.collectd.CollectionWarning: Unexpected exception when collecting SNMP data for interface 172.16.1.134 at location Default.
2017-03-14 09:21:49,038 WARN  [Collectd-Thread-17-of-50] o.o.n.c.CollectableService: Unexpected exception when collecting SNMP data for interface 172.16.1.136 at location Default.
org.opennms.netmgt.collectd.CollectionWarning: Unexpected exception when collecting SNMP data for interface 172.16.1.136 at location Default.
2017-03-14 09:31:59,080 WARN  [Collectd-Thread-39-of-50] o.o.n.c.CollectableService: Unexpected exception when collecting SNMP data for interface 172.16.1.136 at location Default.
org.opennms.netmgt.collectd.CollectionWarning: Unexpected exception when collecting SNMP data for interface 172.16.1.136 at location Default.
2017-03-14 09:39:17,210 WARN  [Collectd-Thread-20-of-50] o.o.n.c.CollectableService: Unexpected exception when collecting SNMP data for interface 172.16.1.134 at location Default.
org.opennms.netmgt.collectd.CollectionWarning: Unexpected exception when collecting SNMP data for interface 172.16.1.134 at location Default.
2017-03-14 09:42:12,284 WARN  [Collectd-Thread-41-of-50] o.o.n.c.CollectableService: Unexpected exception when collecting SNMP data for interface 172.16.1.136 at location Default.
org.opennms.netmgt.collectd.CollectionWarning: Unexpected exception when collecting SNMP data for interface 172.16.1.136 at location Default.

This message appears randomly from random hosts.
I know about bug:
https://issues.opennms.org/browse/NMS-8620 and
https://issues.opennms.org/browse/NMS-8976

but my opennms version is 19.0.1
I was think about pollers increasing, but nothing changes:
nano /opt/opennms/etc/poller-configuration.xml
<poller-configuration threads="200"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Unexpected exception when collecting SNMP data for interface

Roskens, Ronald-2
You need to examine what that full exception statement is telling you.

> -----Original Message-----
> From: taraska90 [mailto:[hidden email]]
> Sent: Wednesday, March 15, 2017 8:55 AM
> To: [hidden email]
> Subject: [opennms-discuss] Unexpected exception when collecting SNMP
> data for interface
>
> Hello firends,
>
> I have issue with "Unexpected exception"
>
> Here log from collectd.log:
> cat /var/log/opennms/collectd.log  | grep "Unexpected exception"
> org.opennms.netmgt.collectd.CollectionWarning: Unexpected exception
> when collecting SNMP data for interface 172.16.1.137 at location Default.
> 2017-03-14 09:18:47,280 WARN  [Collectd-Thread-27-of-50]
> o.o.n.c.CollectableService: Unexpected exception when collecting SNMP
> data for interface 172.16.1.134 at location Default.
> org.opennms.netmgt.collectd.CollectionWarning: Unexpected exception
> when collecting SNMP data for interface 172.16.1.134 at location Default.
> 2017-03-14 09:21:49,038 WARN  [Collectd-Thread-17-of-50]
> o.o.n.c.CollectableService: Unexpected exception when collecting SNMP
> data for interface 172.16.1.136 at location Default.
> org.opennms.netmgt.collectd.CollectionWarning: Unexpected exception
> when collecting SNMP data for interface 172.16.1.136 at location Default.
> 2017-03-14 09:31:59,080 WARN  [Collectd-Thread-39-of-50]
> o.o.n.c.CollectableService: Unexpected exception when collecting SNMP
> data for interface 172.16.1.136 at location Default.
> org.opennms.netmgt.collectd.CollectionWarning: Unexpected exception
> when collecting SNMP data for interface 172.16.1.136 at location Default.
> 2017-03-14 09:39:17,210 WARN  [Collectd-Thread-20-of-50]
> o.o.n.c.CollectableService: Unexpected exception when collecting SNMP
> data for interface 172.16.1.134 at location Default.
> org.opennms.netmgt.collectd.CollectionWarning: Unexpected exception
> when collecting SNMP data for interface 172.16.1.134 at location Default.
> 2017-03-14 09:42:12,284 WARN  [Collectd-Thread-41-of-50]
> o.o.n.c.CollectableService: Unexpected exception when collecting SNMP
> data for interface 172.16.1.136 at location Default.
> org.opennms.netmgt.collectd.CollectionWarning: Unexpected exception
> when collecting SNMP data for interface 172.16.1.136 at location Default.
>
> This message appears randomly from random hosts.
> I know about bug:
> https://issues.opennms.org/browse/NMS-8620 and
> https://issues.opennms.org/browse/NMS-8976
>
> but my opennms version is 19.0.1
> I was think about pollers increasing, but nothing changes:
> nano /opt/opennms/etc/poller-configuration.xml
> <poller-configuration threads="200" #было 30
>
>
>
> --
> View this message in context:
> http://opennms.530661.n2.nabble.com/Unexpected-exception-when-
> collecting-SNMP-data-for-interface-tp7595077.html
> Sent from the OpenNMS - discuss mailing list archive at Nabble.com.
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most engaging
> tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Please read the OpenNMS Mailing List FAQ:
> http://www.opennms.org/index.php/Mailing_List_FAQ
>
> opennms-discuss mailing list
>
> To *unsubscribe* or change your subscription options, see the bottom of
> this page:
> https://lists.sourceforge.net/lists/listinfo/opennms-discuss

----------------------------------------------------------------------
This e-mail message is being sent solely for use by the intended recipient(s) and may contain confidential information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by phone or reply by e-mail, delete the original message and destroy all copies. Thank you.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Unexpected exception when collecting SNMP data for interface

taraska90
Okay, how can I do it?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Unexpected exception when collecting SNMP data for interface

Roskens, Ronald-2
You mean something like this?

2017-03-16 17:12:12,484 WARN  [Collectd-Thread-29-of-150] o.o.n.c.CollectableService: run: failed collection for 5880/XXX.XXX.XXX.XXX/SNMP/BasicPackage
2017-03-16 17:12:12,484 WARN  [Collectd-Thread-29-of-150] o.o.n.c.CollectableService: Unexpected exception when collecting SNMP data for interface XXX.XXX.XXX.XXX at location Default.
org.opennms.netmgt.collectd.CollectionWarning: Unexpected exception when collecting SNMP data for interface XXX.XXX.XXX.XXX at location Default.
        at org.opennms.netmgt.collectd.SnmpCollectionSet.collect(SnmpCollectionSet.java:401) ~[opennms-services-19.0.1.jar:?]
        at org.opennms.netmgt.collectd.SnmpCollector.collect(SnmpCollector.java:333) ~[opennms-services-19.0.1.jar:?]
        at org.opennms.netmgt.collectd.CollectionSpecification.collect(CollectionSpecification.java:274) ~[opennms-services-19.0.1.jar:?]
        at org.opennms.netmgt.collectd.CollectableService.doCollection(CollectableService.java:398) ~[opennms-services-19.0.1.jar:?]
        at org.opennms.netmgt.collectd.CollectableService.doRun(CollectableService.java:337) [opennms-services-19.0.1.jar:?]
        at org.opennms.netmgt.collectd.CollectableService.access$200(CollectableService.java:69) [opennms-services-19.0.1.jar:?]
        at org.opennms.netmgt.collectd.CollectableService$1.run(CollectableService.java:315) [opennms-services-19.0.1.jar:?]
        at org.opennms.core.logging.Logging.withPrefix(Logging.java:71) [org.opennms.core.logging-19.0.1.jar:?]
        at org.opennms.netmgt.collectd.CollectableService.run(CollectableService.java:304) [opennms-services-19.0.1.jar:?]
        at org.opennms.netmgt.scheduler.LegacyScheduler$1.run(LegacyScheduler.java:179) [opennms-services-19.0.1.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
        at org.opennms.core.concurrent.LogPreservingThreadFactory$3.run(LogPreservingThreadFactory.java:124) [opennms-util-19.0.1.jar:?]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
Caused by: java.util.concurrent.ExecutionException: java.lang.Exception: Timeout retrieving SnmpCollectors for XXX.XXX.XXX.XXX for / XXX.XXX.XXX.XXX: SnmpCollectors for XXX.XXX.XXX.XXX: snmpTimeoutError for: /XXX.XXX.XXX.XXX
        at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) ~[?:1.8.0_121]
        at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895) ~[?:1.8.0_121]
        at org.opennms.netmgt.collectd.SnmpCollectionSet.collect(SnmpCollectionSet.java:376) ~[opennms-services-19.0.1.jar:?]
        ... 13 more
Caused by: java.lang.Exception: Timeout retrieving SnmpCollectors for XXX.XXX.XXX.XXX for /XXX.XXX.XXX.XXX: SnmpCollectors for XXX.XXX.XXX.XXX: snmpTimeoutError for: /XXX.XXX.XXX.XXX
        at org.opennms.netmgt.snmp.SnmpWalker.finish(SnmpWalker.java:173) ~[org.opennms.core.snmp.api-19.0.1.jar:?]
        at org.opennms.netmgt.snmp.SnmpWalker.processError(SnmpWalker.java:161) ~[org.opennms.core.snmp.api-19.0.1.jar:?]
        at org.opennms.netmgt.snmp.SnmpWalker.handleTimeout(SnmpWalker.java:151) ~[org.opennms.core.snmp.api-19.0.1.jar:?]
        at org.opennms.netmgt.snmp.snmp4j.Snmp4JWalker.access$1300(Snmp4JWalker.java:48) ~[org.opennms.core.snmp.implementations.snmp4j-19.0.1.jar:?]
        at org.opennms.netmgt.snmp.snmp4j.Snmp4JWalker$Snmp4JResponseListener.onResponse(Snmp4JWalker.java:174) ~[org.opennms.core.snmp.implementations.snmp4j-19.0.1.jar:?]
        at org.snmp4j.Snmp$PendingRequest.run(Snmp.java:1828) ~[org.opennms.core.snmp.implementations.snmp4j-19.0.1.jar:?]
        at java.util.TimerThread.mainLoop(Timer.java:555) ~[?:1.8.0_121]
        at java.util.TimerThread.run(Timer.java:505) ~[?:1.8.0_121]


If you can run "snmpwalk -One $IPADDR" against it, what's likely happening here is the remote node's snmp daemon isn't able to respond fast enough for the query that OpenNMS is sending it.

So how would one go about analyzing how long it takes for a snmpwalk to complete?

I would run a tcpdump, and look at what queries are being sent to the remote snmp daemon. (tcpdump -w /tmp/snmp.tcpdump host $IPADDR and port 161), then load that into wireshark to examine it to find out which OIDs are being queried.

Once I have that information, I would time a run of "snmpwalk -One $IPADDR $OIDS" (or snmpbulkwalk) to see how long that takes to complete.

Then I would update the snmp-config for that ip address (or range) so that my timeout was 5-10s greater than the time from the snmpwalk.


If you get a  "No Response" when you run snmpwalk, then you know you need to investigate why the remote snmp daemon isn't responding to queries from your opennms server.


You could also file a bug in the OpenNMS JIRA asking that the "Unexpected exception when collecting SNMP data" log message be changed to a lower log level.

Ron

> -----Original Message-----
> From: taraska90 [mailto:[hidden email]]
> Sent: Thursday, March 16, 2017 1:57 AM
> To: [hidden email]
> Subject: Re: [opennms-discuss] Unexpected exception when collecting
> SNMP data for interface
>
> Okay, how can I do it?
>
>
>
> --
> View this message in context:
> http://opennms.530661.n2.nabble.com/Unexpected-exception-when-
> collecting-SNMP-data-for-interface-tp7595077p7595080.html
> Sent from the OpenNMS - discuss mailing list archive at Nabble.com.
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most engaging
> tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Please read the OpenNMS Mailing List FAQ:
> http://www.opennms.org/index.php/Mailing_List_FAQ
>
> opennms-discuss mailing list
>
> To *unsubscribe* or change your subscription options, see the bottom of
> this page:
> https://lists.sourceforge.net/lists/listinfo/opennms-discuss

----------------------------------------------------------------------
This e-mail message is being sent solely for use by the intended recipient(s) and may contain confidential information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by phone or reply by e-mail, delete the original message and destroy all copies. Thank you.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Loading...