Splunk Search
Highlighted

How to extract fields from key/value, but separated with "|" symbol?

Contributor

Hi,

I have my syslog file writen as the following. I index these events in a syslog sourcetype.
What I need to extract are fields as PacketyType, PacketIndex, SkinTemperature, StepCounter, DELTADISTANCE and so on...

SocketTLMD:  --------------------   Client :1| PacketType : 6 | senderID : 1.0.0.5 | PacketIndex :26| BatteryVoltage :189| SkinTemperature :23.76| RSSI :78| StepCounter :1| FallCounter :0| AlmostFallCounter :0| MobilityIndex :42| userID : 132234 | CRC :202  | DISTANCE: -3825233.931520 | DELTADISTANCE: -0.000000 | DELTACOLARIES: -0.000000 | SPEED: -0.000000 | DELTASTEP: 0 --------------

Could you suggest a rex to extract only one of these fields?
Thanks,
Skender

0 Karma
Highlighted

Re: How to extract fields from key/value, but separated with "|" symbol?

SplunkTrust
SplunkTrust

Hi skender27,

one hint would be extract http://docs.splunk.com/Documentation/Splunk/6.3.0/SearchReference/Extract another hint can be this answer http://answers.splunk.com/answers/214487/can-i-extract-a-field-with-a-regexed-dynamic-field.html .. in your use case you would split the key value pair with the |

cheers, MuS

Highlighted

Re: How to extract fields from key/value, but separated with "|" symbol?

Contributor

Hi MuS,
Thanks a lot.

I'd prefer insert the extraction in props.conf like (example for the PacketType field):

EXTRACT-pcktype = PacketType :  (?P<pcktype>\d+)\s\|

The problem is that I cannot extratc all the fields I need, changing slightly the precedent regex it seems it is not enough...

I will try again!

Thanks anyway,
Skender

0 Karma
Highlighted

Re: How to extract fields from key/value, but separated with "|" symbol?

SplunkTrust
SplunkTrust

try this regex (\w+)[\s:\-\s]*([^|\s]+) .. thanks for the kudos 🙂

0 Karma
Highlighted

Re: How to extract fields from key/value, but separated with "|" symbol?

Esteemed Legend

Try this:

props.conf:

[YourSourcetypeHere]
REPORT-kvps = colon_pipe_kvps

transforms.conf

[colon_pipe_kvps]
FORMAT = $1::$2
MV_ADD = 1
REGEX = ([^\s:\|]+)\s*:\s*([^:\|]*)(?:\s*\||$)
Highlighted

Re: How to extract fields from key/value, but separated with "|" symbol?

Contributor

Hi,

I added the stanzas you suggested in the .conf files, but it is still not working.
Splunk in some way recognizes the JSON input format and so I'd prefer to make use of this...

This is how I see in the events in Splunk:

<30>Oct 8 12:46:19 SocketTLMD: (27082) JSON input: {"Client":1,"PacketType":"6","senderID":"1.0.0.4","PacketIndex":121,"BatteryVoltage":184,"SkinTemperature":22.48,"RSSI":92,"StepCounter":13,"FallCounter":0,"AlmostFallCounter":0,"MobilityIndex":58,"userID":"020901","CRC":33}

0 Karma
Highlighted

Re: How to extract fields from key/value, but separated with "|" symbol?

Esteemed Legend

There were problems with my RegEx but I fixed and updated my answer. Try it again.

0 Karma
Highlighted

Re: How to extract fields from key/value, but separated with "|" symbol?

Esteemed Legend

There was also a mismatch in my KO name so copy EVERYTHING fresh.

0 Karma
Highlighted

Re: How to extract fields from key/value, but separated with "|" symbol?

Esteemed Legend

OK, then you need to switch from pipes to commas so use this RegEx instead:

 REGEX = ([^\s:,]+)\s*:\s*([^:,]*)(?:\s*,|$)
0 Karma
Highlighted

Re: How to extract fields from key/value, but separated with "|" symbol?

Contributor

After I backed-up my .conf files, actually I am resolving it adding each extraction from the syslog:

EXTRACT-Distance = DISTANCE:\s(?<distance>\d+)
EXTRACT-Calories = Calories:\s(?<calories>\d+)
EXTRACT-PktType = PacketType :\s(?<pcktype>\d+)
EXTRACT-UserID = userID :\s(?<userTLMD>\d+)
EXTRACT-DeltaStep = DELTASTEP:\s(?<deltastep>\d+)\s\-
EXTRACT-DeltaDistance = DELTADISTANCE:\s(?<ddistance>\d+)
EXTRACT-DeltaCalories = DELTACOLARIES:\s(?<dcalories>\d+)
EXTRACT-Speed = SPEED:\s(?<speed>\d+)

View solution in original post

0 Karma