How can I break down an event

[ninisimonishvil]

I have an input from app - WEB Input

It extracts last 5 events from webpage every 1 minute. however instead of spitting them into 5 Splunk sees it as 1 event :

განცხადებების სტატუსების ბოლო 5 ცვლილება მიმდინარეობს ხელშეკრულების მომზადება 07.02.2018 16:01 NAT180000701 შემსყიდველი: შპს ,,ბათუმის წყალი\" კატეგორია: 44100000 არ შედგა 07.02.2018 16:01 NAT180001544 შემსყიდველი: ახალციხის მუნიციპალიტეტის მერია კატეგორია: 50100000 გამარჯვებული გამოვლენილია 07.02.2018 16:00 NAT180000701 შემსყიდველი: შპს ,,ბათუმის წყალი\" კატეგორია: 44100000 შერჩევა/შეფასება 07.02.2018 16:00 NAT180000701 შემსყიდველი: შპს ,,ბათუმის წყალი\" კატეგორია: 44100000 წინადადებების მიღება დასრულებულია 07.02.2018 16:00 NAT180001544 შემსყიდველი: ახალციხის მუნიციპალიტეტის მერია კატეგორია: 50100000"

every event starts with date and ends with space followed by 8 consecutive numbers.

I tried to use SHOULD_MERGE and MUST_BREAK AFTER \s\d{8}
Also tried BREAK_ONLY_BEFORE \d{2}[.]\d{2}[.]\d{4} \d{2}:\d{2}

However getting no results.

[Yunagi · Accepted solution]

Have a look at Configure event line breaking.

Event line breaking consists of two steps: 1st) line breaking and 2nd) line merging.

Line breaking is mostly configured by LINE_BREAKER. By default, LINE_BREAKER is the newline character.

Line merging is configured by SHOULD_LINEMERGE=true and a couple of other options like BREAK_ONLY_BEFORE_DATE.

You should be concerned with line breaking. I suggest you try something like this:

LINE_BREAKER = ( \d{8})

[ninisimonishvil]

Tried that too. Still no result.

[Yunagi]

Are you running a single instance of Splunk? Or do you have multiple insances? This configuration (via props.conf) needs to be placed on the instance where the indexing phase happens. That could be a heavy forwarder.

Set SHOULD_LINEMERGE=false (along with the LINE_BREAKER option) and see if that makes a difference.

[ninisimonishvil]

It is a single instance yes and props.conf needs to be placed in application's local folder, since its the application that takes data from website.

[Yunagi]

Try it like this:

[yoursourcetype]
LINE_BREAKER = ( )\d\d\.\d\d\.\d\d\d\d \d\d:\d\d

As you can see it should work: screenshot
(I autotranslated your input file.)

If it still does not work, can you post your props.conf? I would like to see the relevant stanza.

Also, don't forget to restart Splunk after editing configuration files.

[ninisimonishvil]

worked. thanks a lot!