topic Re: Parse Time From Splunk Forwarder Logs when not send in Key=value pair in Getting Data In

Parse Time From Splunk Forwarder Logs when not send in Key=value pair

MayankSplunk — Mon, 24 Nov 2014 20:31:44 GMT

Following are the logs I'm sending to Splunk. Can someone please guide me how to time subtract time if I group by id?
Since I'm not sending time in Key=value pair I don't know what column to pull.

2014-11-24 14:25:29,873 id=98d57314-740f-11e4-95a8-fe0b46e8751a, fileSize=0, duration=0, status=uploading
2014-11-24 14:26:04,931 id=e5772c54-740e-11e4-95a8-fe0b46e8751a, fileSize=0, duration=103000, status=completed

Re: Parse Time From Splunk Forwarder Logs when not send in Key=value pair

norbert_hamel — Mon, 24 Nov 2014 21:58:47 GMT

Hi, I have to admit that I do not really understand the question. What do you want to subtract? Could you drop an example based on the logs?

Re: Parse Time From Splunk Forwarder Logs when not send in Key=value pair

MayankSplunk — Mon, 24 Nov 2014 22:01:23 GMT

Hi, you see time 2014-11-24 14:25:29,873 2014-11-24 14:26:04,931 in two different row. I would like to subtract them.

Re: Parse Time From Splunk Forwarder Logs when not send in Key=value pair

musskopf — Mon, 28 Sep 2020 18:15:32 GMT

The timestamp should be automatic placed inside the field "_time". Splunk store time as seconds, but if you use it names "_time" it'll automatic format for you.

index=bla "your search" | table _time, id

Remember that Splunk stores the Time in seconds, but it'll auto-format to a more human readable if the field name is "_time". For example:

index=bla "your search" | eval time2=_time | table _time, time2, id

But you might be looking to do something like:

index=bla "your search" | transaction id | table _time, id, duration, status

Have a look on the transaction command documentation, there are plenty of option...

Re: Parse Time From Splunk Forwarder Logs when not send in Key=value pair

musskopf — Mon, 24 Nov 2014 22:08:00 GMT

Ok, now things changed a bit... You might are looking for the command transaction, which will add the field duration that is exactly the time from the first event till the last event. But in your example, the IDs are different. Should they be the same?

Re: Parse Time From Splunk Forwarder Logs when not send in Key=value pair

MayankSplunk — Mon, 24 Nov 2014 22:12:38 GMT

Yes, I mistakenly removed third row which had same ID as first. Let me try your approach.

Re: Parse Time From Splunk Forwarder Logs when not send in Key=value pair

MayankSplunk — Mon, 24 Nov 2014 23:09:48 GMT

Sorry but _time is not getting me multiple _time when grouped by id,

(status="uploading" OR status="completed") | transaction id | table id, _time

above query gives only one result time, am I doing something wrong?

P.S : Don't have enuf point to reply to your answer below

Re: Parse Time From Splunk Forwarder Logs when not send in Key=value pair

musskopf — Mon, 24 Nov 2014 23:13:56 GMT

Try using:

(status="uploading" OR status="completed") | transaction id mvlist=t | table id, _time

(status="uploading" OR status="completed") | transaction id mvlist=_time | table id, _time

Re: Parse Time From Splunk Forwarder Logs when not send in Key=value pair

MayankSplunk — Mon, 24 Nov 2014 23:18:57 GMT

Using t give null and _time still pulls only 1 item

Re: Parse Time From Splunk Forwarder Logs when not send in Key=value pair

musskopf — Mon, 24 Nov 2014 23:25:57 GMT

If you take the transaction command off, is your search returning duplicated values on the id column? The transaction command should just "group" all the events having the same "id" and create multi-value fields for cases where the same field has distinct values.

http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Transaction

Re: Parse Time From Splunk Forwarder Logs when not send in Key=value pair

MayankSplunk — Mon, 24 Nov 2014 23:37:03 GMT

transaction is working fine because it gives me multiple eventCount based on # of similar IDs

Problem is _time is always of the first item that we have in group while using transaction.

Re: Parse Time From Splunk Forwarder Logs when not send in Key=value pair

musskopf — Mon, 24 Nov 2014 23:40:17 GMT

ok, so add | eval eventTime=_time before the transaction command:

(status="uploading" OR status="completed") | eval eventTime=_time | transaction id | table id, _time, eventTime

But as I mentioned, Splunk will show it as seconds, you should be able to convert it back using somenthing like: | convert timeformat="%F %T" ctime(eventTime) at the end.

Re: Parse Time From Splunk Forwarder Logs when not send in Key=value pair

MayankSplunk — Tue, 25 Nov 2014 15:17:54 GMT

eval eventTime=_time did the trick , thanks for all the help

Re: Parse Time From Splunk Forwarder Logs when not send in Key=value pair

MayankSplunk — Tue, 25 Nov 2014 16:53:13 GMT

@musskopf I have follow up question for same query - can you take a look

http://answers.splunk.com/answers/194858/remove-unique-rows-from-table.html