Getting Data In

How to configure Splunk to recognize the correct timestamp from tshark data output and index all remaining fields properly?

mikefoti
Communicator

I'm using tshark to carve out and send specific fields to a txt file, in hopes splunk will index it properly. But not only is the date field misinterpreted, all data after the data is ignored (not indexed).

This is what is what I see in the Data Preview when I attempt to create a new source type:

1 2/3/02 2:13:22.116 PM Apr  2, 2015 13:46:32.116002000 192.168.2.64    59356   123.123.123.1   443

2 2/3/02 2:13:22.398 PM Apr  2, 2015 13:46:32.398707000 192.168.130.207 57524   122.122.122.1   443

3 2/3/02 2:13:22.415 PM Apr  2, 2015 13:46:32.415975000 192.168.231.22  51513   121.121.121.1   443

Note the interpreted date is 2/3/2002 vs the actual date in 2015.

Using notepad++ I can see the data elements are separated by TABs. It seems once splunk thinks it did a fine job on the date, it believes the 1st tab (just after the date/time) is the end of line so, it seems, it ignores everything until it sees the next date.

Having read several questions/answers leads me to believe data from tshark should be easily recognized and indexed.

my tshark command is:

tshark -r myInputFile.raw tcp.flags.syn eq 0x1 -T fields -e frame.time -e ip.src -e tcp.srcport -e ip.dst -e tcp.dstport > myOutputFile.txt

My Question - how can I get each data element to be indexed properly?

0 Karma
1 Solution

esix_splunk
Splunk Employee
Splunk Employee

Since you have multiple time stamps in the event, you need select the right one for Splunk to recognize.

http://docs.splunk.com/Documentation/Splunk/6.2.2/Data/Configuretimestamprecognition

Your format should be the :

Apr  2, 2015 13:46:32.116002000

Portion from your log.

So in your props, your timestamp format should be something like:

TIME_FORMAT =%b, %Y %H:%M:%S.%9N
TIME_PREFIX= \t

I'm not sure if there is a tab, you could also try

TIME_PREFIX = (AM\s|PM\s)

That looks for AM OR PM (not its followed by a space. That might be a tab in your log..)

View solution in original post

esix_splunk
Splunk Employee
Splunk Employee

Since you have multiple time stamps in the event, you need select the right one for Splunk to recognize.

http://docs.splunk.com/Documentation/Splunk/6.2.2/Data/Configuretimestamprecognition

Your format should be the :

Apr  2, 2015 13:46:32.116002000

Portion from your log.

So in your props, your timestamp format should be something like:

TIME_FORMAT =%b, %Y %H:%M:%S.%9N
TIME_PREFIX= \t

I'm not sure if there is a tab, you could also try

TIME_PREFIX = (AM\s|PM\s)

That looks for AM OR PM (not its followed by a space. That might be a tab in your log..)

mikefoti
Communicator

Esix_splunk,
Thanks... once I added %d after %b, it worked.

TIME_FORMAT =%b %d, %Y %H:%M:%S.%9N

Get Updates on the Splunk Community!

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...

New! Splunk Observability Search Enhancements for Splunk APM Services/Traces and ...

Regardless of where you are in Splunk Observability, you can search for relevant APM targets including service ...

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...