topic Re: How to extract fields from my log when every record in the log has different position of the field? in Getting Data In

How to extract fields from my log when every record in the log has different position of the field?

varunchhabra — Wed, 03 May 2017 12:53:46 GMT

I have a log file that contains time stamped events. The type of action done is defined by the string parameter like : FunctionCalled.
I want to extract the action field and use it. But the problem is that the keywords are not following any pattern. It could be like:

TimeStamp1 FunctionCalled
TimeStamp2 ABC::CDE<>FunctionCalledAgain

In such case, the neither the regex nor delimited pattern helping me to get the field.

On the side note, Is it necessary for the log to be ordered so as to apply generic statistical functions?

Plz help.

Re: How to extract fields from my log when every record in the log has different position of the field?

niketn — Wed, 03 May 2017 13:16:06 GMT

How many of above patterns may exist? If there is no way for you to come up with regular expression,

Option 1
You create separate Field Extractions for FunctionalCalled say FunctionalCalled1 and FunctionalCalled2 etc and then use the following in your query:

<YourBaseSearch> FunctionalCalled1="FunctionalCalled" OR FunctionalCalled2="FunctionalCalled"

Option 2
If you do not want to perform field extraction then in base search you can use "FunctionalCalled" and in your subsequent pipe you can use searchmatch to create your own field with eval

| eval FunctionalCalled1=case(searchmatch("FunctionalCalled"),"FunctionalCalled",true(),"Other")

Ideally, interesting fields should be a key value pair, you should also read Logging best practices in order to better utilize Splunk's capabilities: http://dev.splunk.com/view/logging/SP-CAAAFCK

Re: How to extract fields from my log when every record in the log has different position of the field?

gcusello — Wed, 03 May 2017 13:24:02 GMT

Hi varunchhabra,
if your logs have a limited number of types you can build one extraction for each situation (FunctionCalled1, FunctionCalled2, FunctionCalledn) and then configure a calculated field as a coalesce field or an eval command
| eval FunctionCalled=coalesce(FunctionCalled1, FunctionCalled2,...,FunctionCalledn).

Bye.
Giuseppe