topic Re: extracting multiple values for a field from one row(one event) in Splunk Search

extracting multiple values for a field from one row(one event)

smolcj — Wed, 14 Nov 2012 12:26:16 GMT

Hi all,
Regex is troubling me when i have to extract a field compared with previous line. My log is like

Thread Event1 blablabla [something]....completed.
idletime:45.56
Thread Event2 blablabla [something]....completed.no ststistics
Thread Event3 blablabla [something]....completed.no ststistics
Thread Event4 blablabla [something]....completed.no ststistics
Thread Event5 blablabla [something]....completed.
idletime:45.56

how to extract those idle time related with Event1? i have extracted the evantnames as field, but i am not getting any idea to continue the search with a regex.. please help..
Thank you for your time

Question edited

i am sorry that i am confusing you people.. i will try to be more precise about my question.
1.timestamp : [threadname] : threadid :logdetails
2.timestamp : [threadname] : threadid
...........log summary............
Thread Event1 blablabla [something]....completed.no ststistics
Thread Event2 blablabla [something]....completed.
idletime:33
busytime:88
Thread Event3 blablabla [something]....completed.no ststistics
Thread Event4 blablabla [something]....completed.
idletime:55
busytime:99
3.timestamp : [threadname] : threadid :logdetails
4.timestamp : [threadname] : threadid :logdetails

so issue here is, splunk reads only the first field that is it will read the italicized inputs, but i need it to read all the data in BOLD. i know that this issue is because splunk extract only the first value from a line and ignores the repeated ones and here all these information comes under one event and because of that splunk is ignoring the repeated values
so now please help me to solve this issue
thank you

Re: extracting multiple values for a field from one row(one event)

martin_mueller — Wed, 14 Nov 2012 12:48:38 GMT

You can make the regex multiline-capable by adding the flag (?m) at the beginning.

Re: extracting multiple values for a field from one row(one event)

smolcj — Wed, 14 Nov 2012 13:00:02 GMT

All these logs come under one event, so even if i use (?m) only the first event name is extracting

Re: extracting multiple values for a field from one row(one event)

Ayn — Wed, 14 Nov 2012 13:02:40 GMT

That's because you configured your extraction incorrectly. Where and how have you configured the field extraction?

Re: extracting multiple values for a field from one row(one event)

martin_mueller — Wed, 14 Nov 2012 13:04:14 GMT

You might be looking for multi-value fields then: http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/ConfigureSplunktoparsemulti-valuefields

Re: extracting multiple values for a field from one row(one event)

smolcj — Wed, 14 Nov 2012 13:06:57 GMT

AYN, the log is defined so, these part of the log comes under some log summary.. so even if it come in next line, it is of same event details.

Re: extracting multiple values for a field from one row(one event)

bmacias84 — Wed, 14 Nov 2012 15:10:56 GMT

@smolcj, So how does Splunk Break your event or is that one event defined by splunk. I dont see a time stamp?

Re: extracting multiple values for a field from one row(one event)

Rob — Wed, 14 Nov 2012 18:16:50 GMT

@smolcj

Even if this is one single event and not being broken apart you should still be able to extract the idletime as a field.

Thread\sEvent\d.*[\r\n]\s+Idletime\:(?<##myIdleTime>\d+\.\d{2})

(Please remove the hashes from the line above, the formatting on answers doesn't like angle brackets.)

Splunk will look over multiple lines by default so you don't really have to set the (?m) flag.

Based on your comments you are actually looking to extract multiple fields from the multiline event you provided. With that in mind, I have taken the event log snippet and highlighted everything that based on your original question and your comments are what you wish to extract as fields:

Thread Event1 blablabla [something]....completed.

idletime:45.56

Thread Event2 blablabla [something]....completed.no ststistics

Thread Event3 blablabla [something]....completed.no ststistics

Thread Event4 blablabla [something]....completed.no ststistics

Thread Event5 blablabla [something]....completed.

idletime:45.56

The previous regex already extracts the idletime for you across multiple lines. Based on your regex in the comments;

(?i)Threads(?P<fieldname1>[^s]+)sblablablas[(?P<fieldname2>[^]]+)]...completed.nosstatistics

you would also like to extract another 2 fields. I am not sure if there were any typo's when you included that in your comments and the formatting rules made it a bit odd, but here is a corrected regex to match what you posted:

Thread\s(?<##fieldname1>[^\s]+).*?\[(?<##fieldname2>[^\]]+)\]\.{4}completed.no\sststistics

Lastly if you want to change that last regex to get the event number and "something" then you may want to change the above regex to:

Thread\s(?<##fieldname1>[^\s]+).*?\[(?<##fieldname2>[^\]]+)\]

Re: extracting multiple values for a field from one row(one event)

martin_mueller — Wed, 14 Nov 2012 20:08:59 GMT

You can use the < character through explicitly typing the lt entity.

Re: extracting multiple values for a field from one row(one event)

smolcj — Thu, 15 Nov 2012 12:34:01 GMT

The real issue is splunk reads only the first occurence of the event1 the second line is not reading as because it is in same event, how can i extract all the events?
please help

Re: extracting multiple values for a field from one row(one event)

smolcj — Thu, 15 Nov 2012 12:39:03 GMT

yes it have a time stamp
log is like
timestamp severity processid Event1 bla blalog info
******** log statistics*******
Thread Event1 blablabla [something]....completed.
idletime:45.56
Thread Event2 blablabla [something]....completed.no ststistics
Thread Event3 blablabla [something]....completed.no ststistics
Thread Event4 blablabla [something]....completed.
idletime:45.56
timestamp severity processid Event1 bla blalog info
timestamp severity processid Event1 bla blalog info
timestamp severity processid Event1 bla blalog info

Re: extracting multiple values for a field from one row(one event)

Ayn — Thu, 15 Nov 2012 12:39:33 GMT

Well like I previously asked, how are you currently extracting it?

Re: extracting multiple values for a field from one row(one event)

smolcj — Thu, 15 Nov 2012 12:48:36 GMT

i am extracting "Event" and something as 2 separate fields
regex is like (?i)Thread\s(?P[^\s]+)\sblablabla\s[(?P[^]]+)]...completed.no\sstatistics
like this i am trying to extract 2 fields but only the first occurrence is extracting. shall i use TOKENIZER but i guess it is for separating same field elements.. i have already a field named event and it is extracting correctly bot not these thread fields

Re: extracting multiple values for a field from one row(one event)

smolcj — Thu, 15 Nov 2012 13:12:35 GMT

the regex i extracted is working fine in http://gskinner.com/RegExr/

somebody please tell me how to extract multivalue from same line or event(in splunk language)..
another issue is enabling or disabling something in general will make my some other extractions in trouble.. so suggest something which i can use only for this particular field
thank you

Re: extracting multiple values for a field from one row(one event)

sruthy — Thu, 15 Nov 2012 13:22:52 GMT

regExr is to create regular expression . from what u mentioned, the problem is with breakdown of events. did u define it precisely while creating sourcetype or uploading datainputs?

Re: extracting multiple values for a field from one row(one event)

smolcj — Thu, 15 Nov 2012 13:24:32 GMT

yes, it is fine.. these lines comes under an even. but i have to extract it and show separately as a table..
thank you

Re: extracting multiple values for a field from one row(one event)

Rob — Thu, 15 Nov 2012 19:06:46 GMT

@martin_mueller: I did try that but it still formats the string badly. 😞

@smolcj: The regex I provided earlier would have worked in splunk to give you the first value that you were looking for. I will update my answer to reflect the regex from your comments here.

Re: extracting multiple values for a field from one row(one event)

smolcj — Wed, 21 Nov 2012 11:53:35 GMT

what is the splunk property that ignores the multiple occurences of fieldvalues in splunk.. i think that is the reason for my issue
anybody please help
thank you

Re: extracting multiple values for a field from one row(one event)

smolcj — Wed, 21 Nov 2012 12:05:33 GMT

Do i have to disable something to prevent ignoring multiple values from same event in splunk?

Re: extracting multiple values for a field from one row(one event)

Rob — Wed, 21 Nov 2012 18:25:51 GMT

@smolcj this is a bit off toppic from this question and you probably want to ask each of your questions separately so they do not get confusing.

That said, the search command you are looking for is 'dedup'