Syslog no longer matching post upgrade from 19.0.x to 20.0.1

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Syslog no longer matching post upgrade from 19.0.x to 20.0.1

Madden, Joe

Hi All,

 

We use a lot of syslog messages which we matching on process match, and Severity.

 

These configurations worked on v19 but not v20. We did update to 20.0.1 to fix the syslogd-configuration.xml re-ordering but the matches which worked before, no longer work.

 

Please see an example syslog message (Below and attached as image):

 

<14>Jul 18 14:31:51 HAL HAL_ASE[-]: Logstash is running ok 18/07/2017 14:31:51.25

 

 

Our syslog configuration is like so:

 

<?xml version="1.0"?>

 

<syslogd-configuration>

    <configuration

            syslog-port="10514"

            new-suspect-on-message="false"

            parser="org.opennms.netmgt.syslogd.CustomSyslogParser"

            forwarding-regexp="^.*\s(19|20)\d\d([-/.])(0[1-9]|1[012])\2(0[1-9]|[12][0-9]|3[01])(\s+)(\S+)(\s)(\S.+)"

            matching-group-host="6"

            matching-group-message="8"

            discard-uei="DISCARD-MATCHING-MESSAGES"

            />

 

    <import-file>syslog/Custom.syslog.xml</import-file>

    <import-file>syslog/ApacheHTTPD.syslog.xml</import-file>

    <import-file>syslog/LinuxKernel.syslog.xml</import-file>

    <import-file>syslog/NetgearProsafeSmartSwitch.syslog.xml</import-file>

    <import-file>syslog/OpenSSH.syslog.xml</import-file>

    <import-file>syslog/OpenWrt.syslog.xml</import-file>

    <import-file>syslog/Procmail.syslog.xml</import-file>

    <import-file>syslog/Postfix.syslog.xml</import-file>

    <import-file>syslog/Sudo.syslog.xml</import-file>

 

 

 

</syslogd-configuration>

 

File: syslog/Custom.syslog.xml

 

<syslogd-configuration-group>

    <ueiList>

       <ueiMatch>

            <process-match expression="^HAL_ASE$" />

            <match type="regex" expression="^((.+?) (.*))\r?\n?$"/>

            <uei>mottmac.com/syslog/Logstash/informational</uei>

            <severity>Info</severity>

        </ueiMatch>

    </ueiList>

</syslogd-configuration-group>

 

 

Any ideas why these would no longer match?

 

Thanks

 

Joe


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

SyslogMessage.PNG (11K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Syslog no longer matching post upgrade from 19.0.x to 20.0.1

Madden, Joe

Hi All,

 

I am still trying to get to the bottom of this if anyone has any ideas.

 

 

Cheers

 

Joe.

 

From: Madden, Joe [mailto:[hidden email]]
Sent: 18 July 2017 16:11
To: General OpenNMS Discussion <[hidden email]>
Subject: [opennms-discuss] Syslog no longer matching post upgrade from 19.0.x to 20.0.1

 

Hi All,

 

We use a lot of syslog messages which we matching on process match, and Severity.

 

These configurations worked on v19 but not v20. We did update to 20.0.1 to fix the syslogd-configuration.xml re-ordering but the matches which worked before, no longer work.

 

Please see an example syslog message (Below and attached as image):

 

<14>Jul 18 14:31:51 HAL HAL_ASE[-]: Logstash is running ok 18/07/2017 14:31:51.25

 

 

Our syslog configuration is like so:

 

<?xml version="1.0"?>

 

<syslogd-configuration>

    <configuration

            syslog-port="10514"

            new-suspect-on-message="false"

            parser="org.opennms.netmgt.syslogd.CustomSyslogParser"

            forwarding-regexp="^.*\s(19|20)\d\d([-/.])(0[1-9]|1[012])\2(0[1-9]|[12][0-9]|3[01])(\s+)(\S+)(\s)(\S.+)"

            matching-group-host="6"

            matching-group-message="8"

            discard-uei="DISCARD-MATCHING-MESSAGES"

            />

 

    <import-file>syslog/Custom.syslog.xml</import-file>

    <import-file>syslog/ApacheHTTPD.syslog.xml</import-file>

    <import-file>syslog/LinuxKernel.syslog.xml</import-file>

    <import-file>syslog/NetgearProsafeSmartSwitch.syslog.xml</import-file>

    <import-file>syslog/OpenSSH.syslog.xml</import-file>

    <import-file>syslog/OpenWrt.syslog.xml</import-file>

    <import-file>syslog/Procmail.syslog.xml</import-file>

    <import-file>syslog/Postfix.syslog.xml</import-file>

    <import-file>syslog/Sudo.syslog.xml</import-file>

 

 

 

</syslogd-configuration>

 

File: syslog/Custom.syslog.xml

 

<syslogd-configuration-group>

    <ueiList>

       <ueiMatch>

            <process-match expression="^HAL_ASE$" />

            <match type="regex" expression="^((.+?) (.*))\r?\n?$"/>

            <uei>mottmac.com/syslog/Logstash/informational</uei>

            <severity>Info</severity>

        </ueiMatch>

    </ueiList>

</syslogd-configuration-group>

 

 

Any ideas why these would no longer match?

 

Thanks

 

Joe


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Syslog no longer matching post upgrade from 19.0.x to 20.0.1

Cyrille Bollu
Hi Joe,

Usually, turning syslogd's log level to TRACE (in file log4j2.xml) shows enough information to debug such issue.

Br,

Cyrille

2017-07-23 11:15 GMT+02:00 Madden, Joe <[hidden email]>:

Hi All,

 

I am still trying to get to the bottom of this if anyone has any ideas.

 

 

Cheers

 

Joe.

 

From: Madden, Joe [mailto:[hidden email]]
Sent: 18 July 2017 16:11
To: General OpenNMS Discussion <[hidden email]>
Subject: [opennms-discuss] Syslog no longer matching post upgrade from 19.0.x to 20.0.1

 

Hi All,

 

We use a lot of syslog messages which we matching on process match, and Severity.

 

These configurations worked on v19 but not v20. We did update to 20.0.1 to fix the syslogd-configuration.xml re-ordering but the matches which worked before, no longer work.

 

Please see an example syslog message (Below and attached as image):

 

<14>Jul 18 14:31:51 HAL HAL_ASE[-]: Logstash is running ok 18/07/2017 14:31:51.25

 

 

Our syslog configuration is like so:

 

<?xml version="1.0"?>

 

<syslogd-configuration>

    <configuration

            syslog-port="10514"

            new-suspect-on-message="false"

            parser="org.opennms.netmgt.syslogd.CustomSyslogParser"

            forwarding-regexp="^.*\s(19|20)\d\d([-/.])(0[1-9]|1[012])\2(0[1-9]|[12][0-9]|3[01])(\s+)(\S+)(\s)(\S.+)"

            matching-group-host="6"

            matching-group-message="8"

            discard-uei="DISCARD-MATCHING-MESSAGES"

            />

 

    <import-file>syslog/Custom.syslog.xml</import-file>

    <import-file>syslog/ApacheHTTPD.syslog.xml</import-file>

    <import-file>syslog/LinuxKernel.syslog.xml</import-file>

    <import-file>syslog/NetgearProsafeSmartSwitch.syslog.xml</import-file>

    <import-file>syslog/OpenSSH.syslog.xml</import-file>

    <import-file>syslog/OpenWrt.syslog.xml</import-file>

    <import-file>syslog/Procmail.syslog.xml</import-file>

    <import-file>syslog/Postfix.syslog.xml</import-file>

    <import-file>syslog/Sudo.syslog.xml</import-file>

 

 

 

</syslogd-configuration>

 

File: syslog/Custom.syslog.xml

 

<syslogd-configuration-group>

    <ueiList>

       <ueiMatch>

            <process-match expression="^HAL_ASE$" />

            <match type="regex" expression="^((.+?) (.*))\r?\n?$"/>

            <uei>mottmac.com/syslog/Logstash/informational</uei>

            <severity>Info</severity>

        </ueiMatch>

    </ueiList>

</syslogd-configuration-group>

 

 

Any ideas why these would no longer match?

 

Thanks

 

Joe


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Syslog no longer matching post upgrade from 19.0.x to 20.0.1

Seibold, Michael-2
In reply to this post by Madden, Joe

Hi Joe,

 

maybe you will find some hints in the release notes - I believe there was a major change,  probably there is a new syslog parser configured by default and some possibility to change back to the old one.

 

-Michael

 

 

Von: Madden, Joe [mailto:[hidden email]]
Gesendet: Sonntag, 23. Juli 2017 11:16
An: General OpenNMS Discussion <[hidden email]>
Betreff: Re: [opennms-discuss] Syslog no longer matching post upgrade from 19.0.x to 20.0.1

 

Hi All,

 

I am still trying to get to the bottom of this if anyone has any ideas.

 

 

Cheers

 

Joe.

 

From: Madden, Joe [mailto:[hidden email]]
Sent: 18 July 2017 16:11
To: General OpenNMS Discussion <[hidden email]>
Subject: [opennms-discuss] Syslog no longer matching post upgrade from 19.0.x to 20.0.1

 

Hi All,

 

We use a lot of syslog messages which we matching on process match, and Severity.

 

These configurations worked on v19 but not v20. We did update to 20.0.1 to fix the syslogd-configuration.xml re-ordering but the matches which worked before, no longer work.

 

Please see an example syslog message (Below and attached as image):

 

<14>Jul 18 14:31:51 HAL HAL_ASE[-]: Logstash is running ok 18/07/2017 14:31:51.25

 

 

Our syslog configuration is like so:

 

<?xml version="1.0"?>

 

<syslogd-configuration>

    <configuration

            syslog-port="10514"

            new-suspect-on-message="false"

            parser="org.opennms.netmgt.syslogd.CustomSyslogParser"

            forwarding-regexp="^.*\s(19|20)\d\d([-/.])(0[1-9]|1[012])\2(0[1-9]|[12][0-9]|3[01])(\s+)(\S+)(\s)(\S.+)"

            matching-group-host="6"

            matching-group-message="8"

            discard-uei="DISCARD-MATCHING-MESSAGES"

            />

 

    <import-file>syslog/Custom.syslog.xml</import-file>

    <import-file>syslog/ApacheHTTPD.syslog.xml</import-file>

    <import-file>syslog/LinuxKernel.syslog.xml</import-file>

    <import-file>syslog/NetgearProsafeSmartSwitch.syslog.xml</import-file>

    <import-file>syslog/OpenSSH.syslog.xml</import-file>

    <import-file>syslog/OpenWrt.syslog.xml</import-file>

    <import-file>syslog/Procmail.syslog.xml</import-file>

    <import-file>syslog/Postfix.syslog.xml</import-file>

    <import-file>syslog/Sudo.syslog.xml</import-file>

 

 

 

</syslogd-configuration>

 

File: syslog/Custom.syslog.xml

 

<syslogd-configuration-group>

    <ueiList>

       <ueiMatch>

            <process-match expression="^HAL_ASE$" />

            <match type="regex" expression="^((.+?) (.*))\r?\n?$"/>

            <uei>mottmac.com/syslog/Logstash/informational</uei>

            <severity>Info</severity>

        </ueiMatch>

    </ueiList>

</syslogd-configuration-group>

 

 

Any ideas why these would no longer match?

 

Thanks

 

Joe


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Syslog no longer matching post upgrade from 19.0.x to 20.0.1

Seth Leger-2
In reply to this post by Madden, Joe
Hi Joe,

I looked at the changes that I made to the syslog parser and couldn't
see any particular reason why your config would not work. There were a
variety of bugfixes that went into 20.0.0.

As far as the new parser, there is a new parser (RadixTreeSyslogParser)
but it is not configured as the default yet. However, it is more
functional than the other parsers so it may be switched to the default
in a future release.

I would follow Cyrille's advice and turn the logging up, it should give
you more details about the parsing inside CustomSyslogParser. Or you
could give the new RadixTreeSyslogParser a whirl. :)

Seth Leger
The OpenNMS Group


On 7/23/17 5:15 AM, Madden, Joe wrote:

> Hi All,
>
>  
>
> I am still trying to get to the bottom of this if anyone has any ideas.
>
>  
>
>  
>
> Cheers
>
>  
>
> Joe.
>
>  
>
> *From:*Madden, Joe [mailto:[hidden email]]
> *Sent:* 18 July 2017 16:11
> *To:* General OpenNMS Discussion <[hidden email]>
> *Subject:* [opennms-discuss] Syslog no longer matching post upgrade from
> 19.0.x to 20.0.1
>
>  
>
> Hi All,
>
>  
>
> We use a lot of syslog messages which we matching on process match, and
> Severity.
>
>  
>
> These configurations worked on v19 but not v20. We did update to 20.0.1
> to fix the syslogd-configuration.xml re-ordering but the matches which
> worked before, no longer work.
>
>  
>
> Please see an example syslog message (Below and attached as image):
>
>  
>
> <14>Jul 18 14:31:51 HAL HAL_ASE[-]: Logstash is running ok 18/07/2017
> 14:31:51.25
>
>  
>
>  
>
> Our syslog configuration is like so:
>
>  
>
> <?xml version="1.0"?>
>
>  
>
> <syslogd-configuration>
>
>     <configuration
>
>             syslog-port="10514"
>
>             new-suspect-on-message="false"
>
>             parser="org.opennms.netmgt.syslogd.CustomSyslogParser"
>
>            
> forwarding-regexp="^.*\s(19|20)\d\d([-/.])(0[1-9]|1[012])\2(0[1-9]|[12][0-9]|3[01])(\s+)(\S+)(\s)(\S.+)"
>
>             matching-group-host="6"
>
>             matching-group-message="8"
>
>             discard-uei="DISCARD-MATCHING-MESSAGES"
>
>             />
>
>  
>
>     <import-file>syslog/Custom.syslog.xml</import-file>
>
>     <import-file>syslog/ApacheHTTPD.syslog.xml</import-file>
>
>     <import-file>syslog/LinuxKernel.syslog.xml</import-file>
>
>     <import-file>syslog/NetgearProsafeSmartSwitch.syslog.xml</import-file>
>
>     <import-file>syslog/OpenSSH.syslog.xml</import-file>
>
>     <import-file>syslog/OpenWrt.syslog.xml</import-file>
>
>     <import-file>syslog/Procmail.syslog.xml</import-file>
>
>     <import-file>syslog/Postfix.syslog.xml</import-file>
>
>     <import-file>syslog/Sudo.syslog.xml</import-file>
>
>  
>
>  
>
>  
>
> </syslogd-configuration>
>
>  
>
> File: syslog/Custom.syslog.xml
>
>  
>
> <syslogd-configuration-group>
>
>     <ueiList>
>
>        <ueiMatch>
>
>             <process-match expression="^HAL_ASE$" />
>
>             <match type="regex" expression="^((.+?) (.*))\r?\n?$"/>
>
>             <uei>mottmac.com/syslog/Logstash/informational</uei>
>
>             <severity>Info</severity>
>
>         </ueiMatch>
>
>     </ueiList>
>
> </syslogd-configuration-group>
>
>  
>
>  
>
> Any ideas why these would no longer match?
>
>  
>
> Thanks
>
>  
>
> Joe
>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>
>
>
> _______________________________________________
> Please read the OpenNMS Mailing List FAQ:
> http://www.opennms.org/index.php/Mailing_List_FAQ
>
> opennms-discuss mailing list
>
> To *unsubscribe* or change your subscription options, see the bottom of this page:
> https://lists.sourceforge.net/lists/listinfo/opennms-discuss
>


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Syslog no longer matching post upgrade from 19.0.x to 20.0.1

Madden, Joe
Hi All,

Thanks for your help.

Adding trace helped me find out the issue.

It looks like some of the severity's I had where wrong on the incoming messages. (Was there a bug fix on the severity?)

As well as syslog configuration being wrong. Should have been setup like this:


            syslog-port="10514"
            new-suspect-on-message="false"
            parser="org.opennms.netmgt.syslogd.CustomSyslogParser"
            forwarding-regexp="^((.+?) (.*))\r?\n?$"
            matching-group-host="2"
            matching-group-message="3"
            discard-uei="DISCARD-MATCHING-MESSAGES"


No idea how that happened. I guess I copied the wrong configuration when comparing the new/old ones or something!

Thanks

Joe.


-----Original Message-----
From: Seth Leger [mailto:[hidden email]]
Sent: 24 July 2017 15:45
To: General OpenNMS Discussion <[hidden email]>
Subject: Re: [opennms-discuss] Syslog no longer matching post upgrade from 19.0.x to 20.0.1

Hi Joe,

I looked at the changes that I made to the syslog parser and couldn't see any particular reason why your config would not work. There were a variety of bugfixes that went into 20.0.0.

As far as the new parser, there is a new parser (RadixTreeSyslogParser) but it is not configured as the default yet. However, it is more functional than the other parsers so it may be switched to the default in a future release.

I would follow Cyrille's advice and turn the logging up, it should give you more details about the parsing inside CustomSyslogParser. Or you could give the new RadixTreeSyslogParser a whirl. :)

Seth Leger
The OpenNMS Group


On 7/23/17 5:15 AM, Madden, Joe wrote:

> Hi All,
>
>  
>
> I am still trying to get to the bottom of this if anyone has any ideas.
>
>  
>
>  
>
> Cheers
>
>  
>
> Joe.
>
>  
>
> *From:*Madden, Joe [mailto:[hidden email]]
> *Sent:* 18 July 2017 16:11
> *To:* General OpenNMS Discussion
> <[hidden email]>
> *Subject:* [opennms-discuss] Syslog no longer matching post upgrade
> from 19.0.x to 20.0.1
>
>  
>
> Hi All,
>
>  
>
> We use a lot of syslog messages which we matching on process match,
> and Severity.
>
>  
>
> These configurations worked on v19 but not v20. We did update to
> 20.0.1 to fix the syslogd-configuration.xml re-ordering but the
> matches which worked before, no longer work.
>
>  
>
> Please see an example syslog message (Below and attached as image):
>
>  
>
> <14>Jul 18 14:31:51 HAL HAL_ASE[-]: Logstash is running ok 18/07/2017
> 14:31:51.25
>
>  
>
>  
>
> Our syslog configuration is like so:
>
>  
>
> <?xml version="1.0"?>
>
>  
>
> <syslogd-configuration>
>
>     <configuration
>
>             syslog-port="10514"
>
>             new-suspect-on-message="false"
>
>             parser="org.opennms.netmgt.syslogd.CustomSyslogParser"
>
>            
> forwarding-regexp="^.*\s(19|20)\d\d([-/.])(0[1-9]|1[012])\2(0[1-9]|[12][0-9]|3[01])(\s+)(\S+)(\s)(\S.+)"
>
>             matching-group-host="6"
>
>             matching-group-message="8"
>
>             discard-uei="DISCARD-MATCHING-MESSAGES"
>
>             />
>
>  
>
>     <import-file>syslog/Custom.syslog.xml</import-file>
>
>     <import-file>syslog/ApacheHTTPD.syslog.xml</import-file>
>
>     <import-file>syslog/LinuxKernel.syslog.xml</import-file>
>
>    
> <import-file>syslog/NetgearProsafeSmartSwitch.syslog.xml</import-file>
>
>     <import-file>syslog/OpenSSH.syslog.xml</import-file>
>
>     <import-file>syslog/OpenWrt.syslog.xml</import-file>
>
>     <import-file>syslog/Procmail.syslog.xml</import-file>
>
>     <import-file>syslog/Postfix.syslog.xml</import-file>
>
>     <import-file>syslog/Sudo.syslog.xml</import-file>
>
>  
>
>  
>
>  
>
> </syslogd-configuration>
>
>  
>
> File: syslog/Custom.syslog.xml
>
>  
>
> <syslogd-configuration-group>
>
>     <ueiList>
>
>        <ueiMatch>
>
>             <process-match expression="^HAL_ASE$" />
>
>             <match type="regex" expression="^((.+?) (.*))\r?\n?$"/>
>
>             <uei>mottmac.com/syslog/Logstash/informational</uei>
>
>             <severity>Info</severity>
>
>         </ueiMatch>
>
>     </ueiList>
>
> </syslogd-configuration-group>
>
>  
>
>  
>
> Any ideas why these would no longer match?
>
>  
>
> Thanks
>
>  
>
> Joe
>
>
>
> ----------------------------------------------------------------------
> -------- Check out the vibrant tech community on one of the world's
> most engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>
>
>
> _______________________________________________
> Please read the OpenNMS Mailing List FAQ:
> http://www.opennms.org/index.php/Mailing_List_FAQ
>
> opennms-discuss mailing list
>
> To *unsubscribe* or change your subscription options, see the bottom of this page:
> https://lists.sourceforge.net/lists/listinfo/opennms-discuss
>


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Syslog no longer matching post upgrade from 19.0.x to 20.0.1

Seth Leger-2
FWIW, here's a list of the Syslogd improvements that went in recently.
Most of the work was included in 19.1.0, so a little earlier than 20.0.0.

https://issues.opennms.org/issues/?jql=project%20%3D%20NMS%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20in%20(19.1.1%2C%2019.1.0%2C%2020.0.0)%20AND%20component%20%3D%20%22Event%20Reception%20-%20Syslog%22%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC

-- Seth


On 7/24/17 11:34 AM, Madden, Joe wrote:

> Hi All,
>
> Thanks for your help.
>
> Adding trace helped me find out the issue.
>
> It looks like some of the severity's I had where wrong on the incoming messages. (Was there a bug fix on the severity?)
>
> As well as syslog configuration being wrong. Should have been setup like this:
>
>
>             syslog-port="10514"
>             new-suspect-on-message="false"
>             parser="org.opennms.netmgt.syslogd.CustomSyslogParser"
>             forwarding-regexp="^((.+?) (.*))\r?\n?$"
>             matching-group-host="2"
>             matching-group-message="3"
>             discard-uei="DISCARD-MATCHING-MESSAGES"
>
>
> No idea how that happened. I guess I copied the wrong configuration when comparing the new/old ones or something!
>
> Thanks
>
> Joe.
>
>
> -----Original Message-----
> From: Seth Leger [mailto:[hidden email]]
> Sent: 24 July 2017 15:45
> To: General OpenNMS Discussion <[hidden email]>
> Subject: Re: [opennms-discuss] Syslog no longer matching post upgrade from 19.0.x to 20.0.1
>
> Hi Joe,
>
> I looked at the changes that I made to the syslog parser and couldn't see any particular reason why your config would not work. There were a variety of bugfixes that went into 20.0.0.
>
> As far as the new parser, there is a new parser (RadixTreeSyslogParser) but it is not configured as the default yet. However, it is more functional than the other parsers so it may be switched to the default in a future release.
>
> I would follow Cyrille's advice and turn the logging up, it should give you more details about the parsing inside CustomSyslogParser. Or you could give the new RadixTreeSyslogParser a whirl. :)
>
> Seth Leger
> The OpenNMS Group
>
>
> On 7/23/17 5:15 AM, Madden, Joe wrote:
>> Hi All,
>>
>>  
>>
>> I am still trying to get to the bottom of this if anyone has any ideas.
>>
>>  
>>
>>  
>>
>> Cheers
>>
>>  
>>
>> Joe.
>>
>>  
>>
>> *From:*Madden, Joe [mailto:[hidden email]]
>> *Sent:* 18 July 2017 16:11
>> *To:* General OpenNMS Discussion
>> <[hidden email]>
>> *Subject:* [opennms-discuss] Syslog no longer matching post upgrade
>> from 19.0.x to 20.0.1
>>
>>  
>>
>> Hi All,
>>
>>  
>>
>> We use a lot of syslog messages which we matching on process match,
>> and Severity.
>>
>>  
>>
>> These configurations worked on v19 but not v20. We did update to
>> 20.0.1 to fix the syslogd-configuration.xml re-ordering but the
>> matches which worked before, no longer work.
>>
>>  
>>
>> Please see an example syslog message (Below and attached as image):
>>
>>  
>>
>> <14>Jul 18 14:31:51 HAL HAL_ASE[-]: Logstash is running ok 18/07/2017
>> 14:31:51.25
>>
>>  
>>
>>  
>>
>> Our syslog configuration is like so:
>>
>>  
>>
>> <?xml version="1.0"?>
>>
>>  
>>
>> <syslogd-configuration>
>>
>>     <configuration
>>
>>             syslog-port="10514"
>>
>>             new-suspect-on-message="false"
>>
>>             parser="org.opennms.netmgt.syslogd.CustomSyslogParser"
>>
>>            
>> forwarding-regexp="^.*\s(19|20)\d\d([-/.])(0[1-9]|1[012])\2(0[1-9]|[12][0-9]|3[01])(\s+)(\S+)(\s)(\S.+)"
>>
>>             matching-group-host="6"
>>
>>             matching-group-message="8"
>>
>>             discard-uei="DISCARD-MATCHING-MESSAGES"
>>
>>             />
>>
>>  
>>
>>     <import-file>syslog/Custom.syslog.xml</import-file>
>>
>>     <import-file>syslog/ApacheHTTPD.syslog.xml</import-file>
>>
>>     <import-file>syslog/LinuxKernel.syslog.xml</import-file>
>>
>>    
>> <import-file>syslog/NetgearProsafeSmartSwitch.syslog.xml</import-file>
>>
>>     <import-file>syslog/OpenSSH.syslog.xml</import-file>
>>
>>     <import-file>syslog/OpenWrt.syslog.xml</import-file>
>>
>>     <import-file>syslog/Procmail.syslog.xml</import-file>
>>
>>     <import-file>syslog/Postfix.syslog.xml</import-file>
>>
>>     <import-file>syslog/Sudo.syslog.xml</import-file>
>>
>>  
>>
>>  
>>
>>  
>>
>> </syslogd-configuration>
>>
>>  
>>
>> File: syslog/Custom.syslog.xml
>>
>>  
>>
>> <syslogd-configuration-group>
>>
>>     <ueiList>
>>
>>        <ueiMatch>
>>
>>             <process-match expression="^HAL_ASE$" />
>>
>>             <match type="regex" expression="^((.+?) (.*))\r?\n?$"/>
>>
>>             <uei>mottmac.com/syslog/Logstash/informational</uei>
>>
>>             <severity>Info</severity>
>>
>>         </ueiMatch>
>>
>>     </ueiList>
>>
>> </syslogd-configuration-group>
>>
>>  
>>
>>  
>>
>> Any ideas why these would no longer match?
>>
>>  
>>
>> Thanks
>>
>>  
>>
>> Joe
>>
>>
>>
>> ----------------------------------------------------------------------
>> -------- Check out the vibrant tech community on one of the world's
>> most engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>
>>
>>
>> _______________________________________________
>> Please read the OpenNMS Mailing List FAQ:
>> http://www.opennms.org/index.php/Mailing_List_FAQ
>>
>> opennms-discuss mailing list
>>
>> To *unsubscribe* or change your subscription options, see the bottom of this page:
>> https://lists.sourceforge.net/lists/listinfo/opennms-discuss
>>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________
> Please read the OpenNMS Mailing List FAQ:
> http://www.opennms.org/index.php/Mailing_List_FAQ
>
> opennms-discuss mailing list
>
> To *unsubscribe* or change your subscription options, see the bottom of this page:
> https://lists.sourceforge.net/lists/listinfo/opennms-discuss
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Please read the OpenNMS Mailing List FAQ:
> http://www.opennms.org/index.php/Mailing_List_FAQ
>
> opennms-discuss mailing list
>
> To *unsubscribe* or change your subscription options, see the bottom of this page:
> https://lists.sourceforge.net/lists/listinfo/opennms-discuss
>


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss