Splunk Search

How to configure LINE_BREAKER regex in props.conf?

louieb3
Path Finder

I have a data source that looks like this:

I0908 09:35:18.395637 3109 vdisk_micro_migrate_egroup_op.cc:1075] ...
I0908 09:35:18.395697 3109 vdisk_micro_migrate_egroup_op.cc:77] ...
I0908 09:35:18.395843 3146 egroup_delete_op.cc:52] ...
I0908 09:35:18.399770 3146 disk_manager.cc:1624] ...
I0908 09:35:18.504919 3106 vdisk_distributed_oplog_slave_write_op.cc:516] ...

After forwarding to the indexer, my events contain multiple lines of data - it seems to break after approximately 4000 characters.

I added this in the props.conf for the application that deals with the data:

[storage:log-Info]
SHOULD_LINEMERGE = false
LINE_BREAKER=([\r\n]+)

I know that this is the default setting anyway but it does not seem to be working. How would you go about debugging or is there any log that shows me how the events are being formed? BTW, I also tried changing the LINE_BREAKER regex to

LINE_BREAKER=([\r\n]+[I,W,E,F][0-1][0-9][0-3][0-9]\s[0-2][0-9]:[0-5][0-9]:[0-5][0-9].[0-9]{6})

No luck.

1 Solution

louieb3
Path Finder

As Omid said, the props.conf should be on the indexer, not on the forwarder. To fix the problem, I (with the help of Splunk support) edited props.conf in the $SPLUNK_HOME/etc/system/local/ directory and added :

[storage:log-Info]
SHOULD_LINEMERGE = false

Thank you, Omid, for your help.

View solution in original post

louieb3
Path Finder

As Omid said, the props.conf should be on the indexer, not on the forwarder. To fix the problem, I (with the help of Splunk support) edited props.conf in the $SPLUNK_HOME/etc/system/local/ directory and added :

[storage:log-Info]
SHOULD_LINEMERGE = false

Thank you, Omid, for your help.

landen99
Motivator

To clarify, it should be on the heavy forwarder, not the universal forwarder.

0 Karma

okrabbe_splunk
Splunk Employee
Splunk Employee

Are you not using back slashes?

I test this regex and it seemed to work:

LINE_BREAKER=([\r\n]+)[IWEF]\d+\s[0-2][0-9]:[0-5][0-9]:[0-5][0-9].[0-9]{6}

A good site to test is regexr.com.

Omid

louieb3
Path Finder

Thank you, Omid. I will read through your link and make changes accordingly. I will post the result as soon as I can.

0 Karma

slebbie_splunk
Splunk Employee
Splunk Employee

any update on this loui3b3?

0 Karma

louieb3
Path Finder

Hi slebbie, yes. Please look at the accepted answer at the top of this thread.

0 Karma

okrabbe_splunk
Splunk Employee
Splunk Employee

The bottom line is that think of UF as kind of just a simple forwarding mechanism that sends chunks of data to the indexers. It generally doesn't do anything at the event level. There are some exceptions around csv files and things on windows but for general file monitors this is the case.

okrabbe_splunk
Splunk Employee
Splunk Employee

Louie: I assume you are forwarding using a universal forwarder which is good because most of the time that is the right choice. However, when you forward using a universal forwarder the parsing and indexing happens on the indexer and not the forwarder.

The props.conf configuration you are making is considered part of the parsing pipeline and so it is not done at UF but at the indexer. This link might help: http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F

louieb3
Path Finder

Omid: I tried your regex and got the same result as I was getting.
No such things as silly questions to someone who is still learning. Sourcetype is correct.
I do have props.conf on the forwarder. So you are saying that it needs to be on the indexer? I don't understand why so could you explain please? Thank you.

0 Karma

okrabbe_splunk
Splunk Employee
Splunk Employee

oh also, just to check but is props.conf on the indexer or forwarder? It should be on the indexer.

okrabbe_splunk
Splunk Employee
Splunk Employee

did you try my regex? If you notice they are slightly different. What splunk finds in the first capture group is discarded so if you have the whole timestamp in there it will discard that.

If you see your props.conf settings in btool it is being picked up. Silly question but is the sourcetype correct?

0 Karma

louieb3
Path Finder

hortonew: I am not too familiar with btool but googling showed me that I can do:

splunk cmd btool --app= props list --debug

I ran this and it shows me the contents of the props.conf of my app. Is there another way to check to see if my props.conf is being overridden?

0 Karma

hortonew
Builder

louieb3: did you use btool to verify that props.conf isn't being overridden by another props.conf?

0 Karma

louieb3
Path Finder

Thanks, Omid. Yes, the regex works. I use a tool called RegexBuddy to test regular expressions. However, after I put it into props.conf, it does not seem to do anything. It almost seems to me like props.conf is being bypassed if that makes any sense.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...