Archive
Highlighted

Extract using pairdelim and kvdelim

Motivator

Hi, First time trying to use pardelim and kvdelim and having no luck.

The data looks like this

####<Aug 19, 2016 3:06:01 PM CDT> <Debug> <ucontrol> <betamax-portal1> <managedServer3> <[ACTIVE] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <> <> <1471637161846> <BEA-000000> <com.ucontrol.rest.security.LoggingFilter  - RCVD REST request: 
GET /rest/icontrol/login HTTP/1.1
Host: betamax-portal1:8080
accept-language: en-us
user-agent: Mozilla/5.0 (iPhone; CPU iPhone OS 9_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Mobile/13C75 (2037495216)
x-login: cccc
x-password: ********
accept: application/json, text/plain, */*
x-allownonactivatedlogin: true
referer: http://10.0.12.143:8100/
x-appkey: defaultKey
x-expires: 600000
cookie: _ga=GA1.1.1016043735.1471636804; _gat=1
accept-encoding: gzip, deflate
Via: 1.1 beta.icontrol.com
X-Forwarded-For: 63.239.122.42
X-Forwarded-Host: beta.icontrol.com
X-Forwarded-Server: beta.icontrol.com
Connection: close
expand: sites,instances,points,functions

And the nth iteration of my non working query looks like this. The query executes but I don't see any of the fields.

index=cox sourcetype=wls_managedserver Stuck| kv pairdelim="\n" kvdelim=":"
0 Karma
Highlighted

Re: Extract using pairdelim and kvdelim

Motivator

Actually the data looks like this....#### <[ACTIVE] ExecuteThread: '20' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1470912327355> <[STUCK] ExecuteThread: '18' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "662" seconds working on the request "Workmanager: mediaUpload-workManager, Version: 0, Scheduled=true, Started=true, Started time: 662872 ms
[
POST /cameraProxy/video?token=MTE0NTg5OjExNDU4OS4xNDcwMzgzNzc1OjExNDU4OS4zOjExNDU4OV83ODAzNDM2NTE3X3ZfMTM2NF80MjM4MjUwOTo2MDAyNTM= HTTP/1.1
Content-Type: video/mp4
Authorization: Basic OTQ0YTBjMjUzMDgxOg==
X-TimestampedFile: 160811023334
X-Capture-Time: 160811023334
Date: Thu, 11 Aug 2016 02:33:39 -0800
X-TriggerType: http
X-EventInfo: httpc_1
Via: 1.0 10.210.192.63
X-Forwarded-For: 10.210.192.5
X-Forwarded-Host: 184.185.0.36
X-Forwarded-Server: 10.210.192.63
Connection: Keep-Alive
Content-Length: 1542184
]", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
null>

0 Karma
Highlighted

Re: Extract using pairdelim and kvdelim

Legend

Try this

index=cox sourcetype=wls_managedserver Stuck| extract pairdelim="\n" kvdelim=":"

View solution in original post

0 Karma
Highlighted

Re: Extract using pairdelim and kvdelim

Motivator

Hi Sundareshr!

Do you ever sleep? 🙂 The query executes but I don't see any of the fields in the field list to the left. I really haven't used this before so I'm pretty sure I'm doing something wrong.... 🙂

0 Karma
Highlighted

Re: Extract using pairdelim and kvdelim

Motivator

I'm trying to pull the values for Content-Type but every time I try to use that field it comes up null, even though the data clearly has values

0 Karma
Highlighted

Re: Extract using pairdelim and kvdelim

Legend

🙂 I don't see Content-Type in the sample you provided? What am I not looking at?

One of the things you may already be aware of, the fields on the left, only show if more than 20% events have the field.

0 Karma
Highlighted

Re: Extract using pairdelim and kvdelim

Motivator

Hi Sundareshr,

I think I pasted the wrong one before. Below is the correct one. The search only returns 2 events and both of them have the Content-Type kv.

####<Aug 11, 2016 6:45:27 AM EDT> <Error> <WebLogicServer> <ccivirpxa0712> <managedServer23> <[ACTIVE] ExecuteThread: '20' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1470912327355> <BEA-000337> <[STUCK] ExecuteThread: '18' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "662" seconds working on the request "Workmanager: mediaUpload-workManager, Version: 0, Scheduled=true, Started=true, Started time: 662872 ms
[
POST /cameraProxy/video?token=MTE0NTg5OjExNDU4OS4xNDcwMzgzNzc1OjExNDU4OS4zOjExNDU4OV83ODAzNDM2NTE3X3ZfMTM2NF80MjM4MjUwOTo2MDAyNTM= HTTP/1.1
Content-Type: video/mp4
Authorization: Basic OTQ0YTBjMjUzMDgxOg==
X-TimestampedFile: 160811023334
X-Capture-Time: 160811023334
Date: Thu, 11 Aug 2016 02:33:39 -0800
X-TriggerType: http
X-EventInfo: httpc_1
Via: 1.0 10.210.192.63
X-Forwarded-For: 10.210.192.5
X-Forwarded-Host: 184.185.0.36
X-Forwarded-Server: 10.210.192.63
Connection: Keep-Alive
Content-Length: 1542184
]", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
null> 
Collapse
0 Karma
Highlighted

Re: Extract using pairdelim and kvdelim

Legend

Not sure why that is not working. You can achieve the same result by adding a field transformation (http://docs.splunk.com/Documentation/Splunk/6.4.2/Knowledge/Managefieldtransforms)

Regular Expression: ([\w+-[^:]+):\s+([^\n]+)
Format: $1::$2
0 Karma
Highlighted

Re: Extract using pairdelim and kvdelim

Motivator

so a follow on question. If the field doesn't appear in 20% of the total events (this would be true in this case as there are 1000's of events but only 2 have the Stuck keyword), how can the field be extracted and used?

0 Karma
Highlighted

Re: Extract using pairdelim and kvdelim

Motivator

Oh Goody! TWO things I've not done before 🙂

Ok I've added the field transformation and re-run the search but still no luck. Is there something else I should be doing?

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.