Getting Data In

How to extract fields from my log when every record in the log has different position of the field?

varunchhabra
New Member

I have a log file that contains time stamped events. The type of action done is defined by the string parameter like : FunctionCalled.
I want to extract the action field and use it. But the problem is that the keywords are not following any pattern. It could be like:

TimeStamp1 FunctionCalled
TimeStamp2 ABC::CDE<>FunctionCalledAgain

In such case, the neither the regex nor delimited pattern helping me to get the field.

On the side note, Is it necessary for the log to be ordered so as to apply generic statistical functions?

Plz help.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi varunchhabra,
if your logs have a limited number of types you can build one extraction for each situation (FunctionCalled1, FunctionCalled2, FunctionCalledn) and then configure a calculated field as a coalesce field or an eval command
| eval FunctionCalled=coalesce(FunctionCalled1, FunctionCalled2,...,FunctionCalledn).

Bye.
Giuseppe

0 Karma

niketn
Legend

How many of above patterns may exist? If there is no way for you to come up with regular expression,

Option 1
You create separate Field Extractions for FunctionalCalled say FunctionalCalled1 and FunctionalCalled2 etc and then use the following in your query:

<YourBaseSearch> FunctionalCalled1="FunctionalCalled" OR FunctionalCalled2="FunctionalCalled"

Option 2
If you do not want to perform field extraction then in base search you can use "FunctionalCalled" and in your subsequent pipe you can use searchmatch to create your own field with eval

| eval FunctionalCalled1=case(searchmatch("FunctionalCalled"),"FunctionalCalled",true(),"Other")

Ideally, interesting fields should be a key value pair, you should also read Logging best practices in order to better utilize Splunk's capabilities: http://dev.splunk.com/view/logging/SP-CAAAFCK

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma
Get Updates on the Splunk Community!

The Payment Operations Wake-Up Call: Why Financial Institutions Can't Afford ...

The same scenario plays out across financial institutions daily. A payment system fails at 11:30 AM on a busy ...

Make Your Case: A Ready-to-Send Letter for Getting Approval to Attend .conf25

Hello Splunkers, Want to attend .conf25 in Boston this year but not sure how to convince your manager? We've ...

Community Spotlight: A Splunk Expert's Journey

In the world of data analytics, some journeys leave a lasting impact not only on the individual but on the ...