All Apps and Add-ons
Highlighted

How is linebreaking of events configured when receiving XML logs via REST input?

Esteemed Legend

I am new to REST inputs and am having a very basic problem that I do not see discussed before. I am using http GET and xml for Response type. I am receiving the response just fine but only as 1 giant event. I would like to split events at the "<alarm>" tag (ending at "</alarm>") but I do not see any way to configure this. On a lark, I went ahead and put a props.conf file in the same app directory as this input with the normal settings that would work for a file-based input but they have no effect. How is linbreaking of events controlled for REST inputs?

Highlighted

Re: How is linebreaking of events configured when receiving XML logs via REST input?

Champion

This is lightly documented in the documentation in the section Custom Response Handlers. The response handlers can be found in the bin directory in a file call responsehandlers.py. By the Modular input use DefaultResponseHandler which just pass the REST response to STDOUT. You can change it to any of the ones find the file by added the following setting to the stanza of your input, response_handler= <class>. class is just one of the classes from responsehandlers.py.

From the sounds of it your REST response gives you multiple events in the message which you need to split or else it comes in as one giant event. Looking at responsehandlers.py it looks like you will have to write a little bit of python which should take about 15 lines of code.

Not know how your raw data looks I might write something like this. This was not tested as just wrote of the hip.

class AlarmResponseHandler:

  def __init__(self,**args):
    pass

  def __call__(self, response_object,raw_response_output,response_type,req_args,endpoint):
    #for each line in response.  Assuming it reads like a text file with new line char returns
    for line in raw_response_output:
      # if ',/alarm>' in the line end of event
      if '</alarm>' in line:
        event = event += line
        print_xml_stream(event)
        # clear event for new event
        event = None
Highlighted

Re: How is linebreaking of events configured when receiving XML logs via REST input?

Ultra Champion

Can you post an example of the full XML you receive. I can then help you with a sample custom response handler.

0 Karma
Highlighted

Re: How is linebreaking of events configured when receiving XML logs via REST input?

New Member
<alarm-list>
<alarm>
<assignedBy/>
<assignedTo/>
<custom1/>
<custom2/>
<custom3/>
<custom4/>
<custom5/>
<devId>D344459B39C33B452C53464089C59E78B</devId>
<domain>Place</domain>
<hostname>name</hostname>
<hub>name</hub>
<id>ZC40864030-00086</id>
<level>2</level>
<message>
ssss.exe: Process ssss.exe is not running
</message>
<metId>MF66466D228CC0462DB58C5CA12D2C789</metId>
<nas>servername</nas>
<origin>servername</origin>
<prevLevel>0</prevLevel>
<probe>processes</probe>
<robot>server</robot>
<severity>Warning</severity>
<source>111.111.111.1111</source>
<subsystem>Process</subsystem>
<subsystemId>1.1.2</subsystemId>
<suppressionCount>0</suppressionCount>
<suppressionKey>process/ssss.exe [1]/process_state</suppressionKey>
<timeArrival>2015-06-10T03:59:15-04:00</timeArrival>
<timeOrigin>2015-06-10T03:59:04-04:00</timeOrigin>
<timeReceived>2015-06-10T03:59:15-04:00</timeReceived>
<userTag1>customer</userTag1>
<userTag2>Production</userTag2>
<visible>true</visible>
</alarm>
<alarm>
<assignedBy/>
<assignedTo/>
<custom1/>
<custom2/>
<custom3/>
<custom4/>
<custom5/>
<devId>D1D431F7A35BB0912BB52FDF175FD3307</devId>
<domain>org/domain>
<hostname>serverName</hostname>
<hub>ServerHub</hub>
<id>YN44419525-32263</id>
<level>4</level>
<message>TrackerSvr PMQ: 219 files, expected < 50</message>
<metId>MF580138FE25DA78DF96521183CA3245A</metId>
<nas>IND1109C2NM13_NMS</nas>
<origin>mel063gnh021</origin>
<prevLevel>0</prevLevel>
<probe>dirscan</probe>
<robot>serverName</robot>
<severity>Major</severity>
<source>22.22.22.22</source>
<subsystem>Probe</subsystem>
<subsystemId>1.2.3</subsystemId>
<suppressionCount>1431</suppressionCount>
<suppressionKey>dirscan/TrackerSvr PMQ/number_of_files</suppressionKey>
<timeArrival>2015-06-08T05:13:29-04:00</timeArrival>
<timeOrigin>2015-02-09T12:05:17-05:00</timeOrigin>
<timeReceived>2015-06-08T05:13:29-04:00</timeReceived>
<userTag1>Customer Name</userTag1>
<userTag2>Production</userTag2>
<visible>false</visible>
</alarm>
<alarm-list>
0 Karma
Highlighted

Re: How is linebreaking of events configured when receiving XML logs via REST input?

Ultra Champion

Try something like :

class AlarmHandler:

    def __init__(self,**args):
        pass

    def __call__(self, response_object,raw_response_output,response_type,req_args,endpoint):
        if response_type == "xml": 
            import xml.etree.ElementTree as ET
            alarm_list = ET.fromstring(encodeXMLText(raw_response_output))
            for alarm in alarm_list:
                alarm_xml_str = ET.tostring(alarm, encoding='utf8', method='xml')
                print_xml_stream(alarm_xml_str)               

        else:
            print_xml_stream(raw_response_output) 

alt text

View solution in original post

Highlighted

Re: How is linebreaking of events configured when receiving XML logs via REST input?

New Member

Should I just be adding this to the bottom of responsehandlers.py ?

0 Karma
Highlighted

Re: How is linebreaking of events configured when receiving XML logs via REST input?

Ultra Champion

Yes.you will also see other examples in that file.note , I just gave an example only, may or may not need a little tweaking.

0 Karma
Highlighted

Re: How is linebreaking of events configured when receiving XML logs via REST input?

New Member

Ok just checking. I am trying it now.

0 Karma
Highlighted

Re: How is linebreaking of events configured when receiving XML logs via REST input?

New Member

For some reason when I added AlarmHandler to the bottom of responsehandlers it broke the polling of the REST data. Do I need to add import xml to the top of the config?

0 Karma
Highlighted

Re: How is linebreaking of events configured when receiving XML logs via REST input?

Champion

Shouldn't have to as it imported when the class is called on line 8.

0 Karma