topic Re: Merge logs in Splunk Search

Merge logs

mastoras — Wed, 30 Sep 2020 05:14:21 GMT

Hello team

I would like to merge more events into one, currently my events look like this:

1st part

{"log":"feign.FeignException$NotFound: status 404 reading xxxxx#getContractDataByContractUuidDynamicV1(String,String)\n","stream":"stdout","time":"2020-04-28T06:09:41.253478466Z","kubernetes":{xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}}

source http:xxx-xxx-xxx

2nd part

{"log":"\tat feign.FeignException.clientErrorStatus(FeignException.java:165)\n","stream":"stdout","time":"2020-04-28T06:09:41.253535467Z","xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}}

3rd....nnth parts are following

So in props.conf I created the stanza like this:

[source::http:xxx-xxx-xxx]
SHOULD_LINEMERGE = true
MUST_NOT_BREAK_BEFORE =
MUST_NOT_BREAK_AFTER = feign.FeignException\$NotFound
MUST_BREAK_AFTER = INFO

but still I do not see the events being merged. Any ideas where to check in order to debug?

Thank you

Re: Merge logs

gcusello — Tue, 28 Apr 2020 06:50:16 GMT

Hi @mastoras,
do you want to merge events at index time or at search Time?
in your question you seem to want to do this at index time, but your events seem to be different (every one has fields and a timestamp), and if you merge them at index time, it's more difficoult to use them.
So probably it could be better to index them one by one and correlate them at search time using the methods by Splunk (transaction, stats, etc...).

Ciao.
Giuseppe

Re: Merge logs

mastoras — Tue, 28 Apr 2020 08:51:14 GMT

I wanted to do it at index time yes, the events are all part of the same Java error message 😕