Getting Data In
Highlighted

Should I configure line breaking for a log with XML data on the universal forwarder or indexer?

New Member

I have a universal forwarder on a remote machine that forwards the splunk enterprise instance a log that may include lines that looks like this :

...
2015-06-15 12:00:01 | INFO ...
2015-06-15 12:00:02 | INFO | text  
(info)
(name) marty (/name)
(timestamp) 2015-06-15 12:00:02 (/timestamp)
(/info)
2015-06-15 12:00:03 | INFO ...
...

(** note that at 12:00:02 - that's supposed to be an xml doc )
Using the Splunk web interface (on the receiving side), I added in a sample file, and configured a new sourcetype on the receiving side to break on ^\d{4}-\d{2}-\d{2} (like 2015-06-15), and I could see that this worked based on the sample data that the web tool displayed. ( My goal is to make it so that Splunk doesn't think that the timestamp xml tag indicates a new Splunk event - I want the whole XML file displayed within the same event. )
I started up the forwarder by specifying the correct index and this new sourcetype.
However, I noticed that it still broke up the line around the tag.

Basically,
Can/should line breaks be configured on the receiver or on the forwarder?
Did I miss a step somewhere??
I understand that some of this can be configured through props.conf / transforms.conf, but I don't have the privileges to see these files yet.

Thanks !

0 Karma
Highlighted

Re: Should I configure line breaking for a log with XML data on the universal forwarder or indexer?

New Member

** note that the line at 12:00:02 is actually supposed to be a tiny XML doc. The format is :
(info)
(name) marty (/name)
(timestamp) 2015-06-15 12:00:00 (/timestamp)
(/info)

Guess the website strips off xml stuff...

0 Karma
Highlighted

Re: Should I configure line breaking for a log with XML data on the universal forwarder or indexer?

Explorer

Do you want both time stamps part of one event or do the time stamps indicate new events regardless of whether or not they contain XML?

0 Karma
Highlighted

Re: Should I configure line breaking for a log with XML data on the universal forwarder or indexer?

New Member

The contents of the XML file, including the timestamp tag, should be part of the same event.

0 Karma
Highlighted

Re: Should I configure line breaking for a log with XML data on the universal forwarder or indexer?

SplunkTrust
SplunkTrust

Hi martinh3,

take a look at this great wiki page http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F it explains where the setting must be done; in your use case it is parsing and this will be either on a heavy forwarder or on an indexer

Also make sure the sourcetype in props.conf matches exactly and that your regex works on regex test pages like www.regex101.com

Hope that helps ...

cheers, MuS

Highlighted

Re: Should I configure line breaking for a log with XML data on the universal forwarder or indexer?

New Member

Thanks for responding to both of my questions, MuS !
I basically re-did the same method in my first post, and it worked the second time around.
The only thing I can think of is that my regex was messed up (maybe missed the ^ at the beginning).
Without permissions to view anything ( hence my other post about ownership of the conf files),
I'm limited.

0 Karma