syslog-configuration issue

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

syslog-configuration issue

Madden, Joe
Hi All,

I have an issue with syslog configuration. We have a customized syslog setup which was working on V19 but after an update to v20 it is now broken.

    <ueiMatch>
        <process-match expression="^HAL_ASE\\DbServer" />
        <match type="regex" expression="^((.+?) (.*))\r?\n?$"/>
        <severity>Critical</severity>
        <uei>mottmac.com/syslog/HAL_ASE/critical</uei>
    </ueiMatch>

Internal Exception: org.xml.sax.SAXParseException: cvc-complex-type.2.4.a: Invalid content was found starting with element 'severity'. One of '{"http://xmlns.opennms.org/xsd/config/syslog":uei}' is expected.]

I therefore reordered it as Seth suggested in the OpenNMS chat to

 
    <ueiMatch>
        <process-match expression="^HAL_ASE\\DbServer" />
        <match type="regex" expression="^((.+?) (.*))\r?\n?$"/>
        <uei>mottmac.com/syslog/HAL_ASE/critical</uei>
        <severity>Critical</severity>
    </ueiMatch>


However I now get the following error:

Invalid content was found starting with element 'severity'. One of '{"http://xmlns.opennms.org/xsd/config/syslog":parameter-assignment}' is expected.

I added <parameter-assignment matching-group="0" parameter-name="Match" /> Thinking it was complaing that this was missing but it doesn't seem to make any difference.

Does anyone have any suggestions on what I would need to do to get around this issue. I was starting to wonder if it was complaining about the use of Critical in <severity>Critical</severity>


Cheers

Joe

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: syslog-configuration issue

Seibold, Michael-2
Hi Joe,

I don't have v20 installed so I can't check it ad hoc - just look into the /opt/opennms/share/xsds/syslogd-configuratiopn-types.xsd (at least it's this one in meridian). The syntax of the config file should be declared there.

In Meridian I found for the severity

                <element minOccurs="0" maxOccurs="unbounded" name="severity">
                    <annotation>
                        <documentation>The name of a syslog severity. If present, the severity of
                                       an incoming message must match one of the severities named
                                       by an instance of this tag within the ueiMatch.
                        </documentation>
                    </annotation>
                    <simpleType>
                        <restriction base="string">
                            <pattern value="[Ee]mergency|[Aa]lert|[Cc]ritical|[Ee]rror|[Ww]arning|[Nn]otice|[Ii]nfo|[Dd]ebug"/>
                        </restriction>
                    </simpleType>
                </element>

If it's not declared there than something went wrong...


One hint for performance: if you know the syslog facility for the incoming syslogs then use the facility as the first matching rule and the severity as second matching rule.
Reason: comparing those values is probably realized as an integer compare and should be MUCH faster then a complex regex comparison. Using those two matching rules in front of the regex should probably avoid for the majority of syslogs to ever reach the regex comparison.

We had some SEVERE performance/availability problems when -due to some bigger "events"- a bunch of syslogs where coming in in a short time. After reconfiguring the syslog as described above the performance/availability problems where gone.

-Michael

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: syslog-configuration issue

Madden, Joe
Hi All,

Sorry for taking so long to reply.

I don't have the file: /opt/opennms/share/xsds/syslogd-configuration-types.xsd (I assume the p was a typo)

To be honest we use Logstash to send a few messages a day the performance issue is unlikely to be a problem for us (And it was all working before v20 upgrade) I suspect something in the V20 change has broken it.


<syslogd-configuration-group>
    <ueiList>
        <ueiMatch>
            <process-match expression="^HAL_ASE\\DbServer" />
            <match type="regex" expression="^((.+?) (.*))\r?\n?$"/>
            <uei>mottmac.com/syslog/HAL_ASE/critical</uei>
            <severity>Critical</severity>
        </ueiMatch>
    </ueiList>
</syslogd-configuration-group>

I can't see anything obviously missing/incorrect from this configuration compared to the  XSD.

Are we support to have the XSD in /opt/opennms/share/xsds/ I assumed the ones that were used would be in a jar somewhere.

Thanks

Joe.

-----Original Message-----
From: Seibold, Michael [mailto:[hidden email]]
Sent: 03 July 2017 12:29
To: General OpenNMS Discussion <[hidden email]>
Subject: Re: [opennms-discuss] syslog-configuration issue

Hi Joe,

I don't have v20 installed so I can't check it ad hoc - just look into the /opt/opennms/share/xsds/syslogd-configuratiopn-types.xsd (at least it's this one in meridian). The syntax of the config file should be declared there.

In Meridian I found for the severity

                <element minOccurs="0" maxOccurs="unbounded" name="severity">
                    <annotation>
                        <documentation>The name of a syslog severity. If present, the severity of
                                       an incoming message must match one of the severities named
                                       by an instance of this tag within the ueiMatch.
                        </documentation>
                    </annotation>
                    <simpleType>
                        <restriction base="string">
                            <pattern value="[Ee]mergency|[Aa]lert|[Cc]ritical|[Ee]rror|[Ww]arning|[Nn]otice|[Ii]nfo|[Dd]ebug"/>
                        </restriction>
                    </simpleType>
                </element>

If it's not declared there than something went wrong...


One hint for performance: if you know the syslog facility for the incoming syslogs then use the facility as the first matching rule and the severity as second matching rule.
Reason: comparing those values is probably realized as an integer compare and should be MUCH faster then a complex regex comparison. Using those two matching rules in front of the regex should probably avoid for the majority of syslogs to ever reach the regex comparison.

We had some SEVERE performance/availability problems when -due to some bigger "events"- a bunch of syslogs where coming in in a short time. After reconfiguring the syslog as described above the performance/availability problems where gone.

-Michael

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: syslog-configuration issue

Seibold, Michael-2
Hi Joe,

> I don't have the file: /opt/opennms/share/xsds/syslogd-configuration-types.xsd
>  Are we support to have the XSD in /opt/opennms/share/xsds/ I assumed the ones that were used would be in a jar somewhere.

In the past we had sometimes a strange behaviour after upgrading because our xsds-directory is in another place (linked via softlink) than opennms installer assumes, and he installed the new files in "his" place. So we sometimes had older xsd files in the directory opennms uses and the GUI somtimes produced corrupt config files (by example when editing thresholds).
That said, it should always be checked that those files correspond with the running version of opennms.

> (And it was all working before v20 upgrade) I suspect something in the V20 change has broken it

Benjamin Reed wrote in "This Week in OpenNMS: July 5th, 2017" :
"I fixed the syslogd-configuration.xml parser to honor out-of-order ueimatch fields, like Castor did."

Unfortunally the corresponding bug number is not mentioned.

So probably you are running into this bug or something similar?

-Michael


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: syslog-configuration issue

Madden, Joe
Yeah - I found that bug and reported it via the OpenNMS Chat.

I suspect its related to that - Can I import the updated xsd somehow or can I update to 20.0.1? - It's not on the repo yet.

I reordered it to get around the ordering bug - But maybe this error is from the same thing still?

Joe.
-----Original Message-----
From: Seibold, Michael [mailto:[hidden email]]
Sent: 10 July 2017 11:49
To: General OpenNMS Discussion <[hidden email]>
Subject: Re: [opennms-discuss] syslog-configuration issue

Hi Joe,

> I don't have the file:
> /opt/opennms/share/xsds/syslogd-configuration-types.xsd
>  Are we support to have the XSD in /opt/opennms/share/xsds/ I assumed the ones that were used would be in a jar somewhere.

In the past we had sometimes a strange behaviour after upgrading because our xsds-directory is in another place (linked via softlink) than opennms installer assumes, and he installed the new files in "his" place. So we sometimes had older xsd files in the directory opennms uses and the GUI somtimes produced corrupt config files (by example when editing thresholds).
That said, it should always be checked that those files correspond with the running version of opennms.

> (And it was all working before v20 upgrade) I suspect something in the
> V20 change has broken it

Benjamin Reed wrote in "This Week in OpenNMS: July 5th, 2017" :
"I fixed the syslogd-configuration.xml parser to honor out-of-order ueimatch fields, like Castor did."

Unfortunally the corresponding bug number is not mentioned.

So probably you are running into this bug or something similar?

-Michael


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
Reply | Threaded
Open this post in threaded view
|

Re: syslog-configuration issue

Seibold, Michael-2
> Can I import the updated xsd somehow

Maybe those xsds files are in /var/opennms/xsds on your system - it may depend on the type of OS

Michael


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss