topic Re: How to move raw data with no field assigned to a table? in Splunk Search

How to move raw data with no field assigned to a table?

cdhippen — Sun, 21 Jan 2018 00:40:46 GMT

This might be a really simple question, but I haven't been able to find an answer as of yet. I have some raw data from some events that is for example "(duration 5555ms)" and I want to put that in a "| timechart span=1m count by duration" to create a chart that shows when these events took place and their total duration. There is currently no field set up for duration, it is just raw data. How would I get those numbers into my time chart?

Re: How to move raw data with no field assigned to a table?

acharlieh — Sun, 21 Jan 2018 20:29:06 GMT

The simple answer is extract the data into a field.

The documentation goes into this in great depth and length, about using the field extractor and many other options: http://docs.splunk.com/Documentation/Splunk/7.0.1/Knowledge/Aboutfields

With a quick one-off though you may consider using the rex command to quickly extract the numeric portion and then use it like so:

<base search> | rex "duration (?<duration_ms>\d+)ms" | timechart count min(duration_ms) max(duration_ms) avg(duration_ms) p95(duration_ms)

I should note, I only had your one partial sample to work with, therefore milage may vary but that's the general idea... Build a regular expression to extract the data you want from your events, with appropriate anchoring. If you need help developing regular expressions, I recommend playing with regex101.com or any number of other interactive regex testers.

If you are taking formal courses from Splunk it looks like this is started to be covered in Splunk Fundamentals 2 and later on with Splunk Data Administration.

Re: How to move raw data with no field assigned to a table?

cdhippen — Sun, 21 Jan 2018 21:51:02 GMT

This is great! I took Splunk fundamentals but they didn't go over rex, so while I've seen it before I didn't exactly know how to use it. Also I tried field extraction but got lost. I'll definitely dive into this documentation, much appreciated!

Re: How to move raw data with no field assigned to a table?

acharlieh — Sun, 21 Jan 2018 21:56:24 GMT

The free Splunk Fundamentals 1 course doesn't get into field extraction... but rather assumes fields are already extracted for you. Fundamentals 2 (which is paid training) picks up where Fundamentals 1 leaves off and gets into the basics of field extractions among other topics.

I have to admit I took the older courses and not the Fundmentals series (I don't remember if this was a normal Searching and Reporting class topic, or an Advanced S&R topic, or if rex landed squarely in the Admin course... but different aspects were covered in different parts of my Splunk class journey).