Getting Data In

Use JSON epoch date time instead of index time

mkarimi17
Path Finder

I have a JSON that is for emails like the following:

{   [-] 
     computer:  {   [+] 
    }   
     date:   2018-03-08T11:42:57+00:00      
     event_type_id:  553648152  
     timestamp:  1520509377 
     timestamp_nanoseconds:  893334279  
}

Note: the time above is in UTC.

However, my time is set to PST and so it looks like I'm getting the index time, timestamp of "11/14/17 6:50:49.000 PM"

This is what's in my props.conf:

[cisco:amp:json]
SHOULD_LINEMERGE = true
pulldown_type = 1
category = Splunk App Add-on Builder
LINE_BREAKER = ([\r\n]*)\{\"event_type\"\:
TIME_PREFIX = timestamp:\s*
TIME_FORMAT = %s
KV_MODE = json
TRANSFORMS-amp_hostname = force_amp_hostname
EXTRACT-amp_hostname = \"hostname\"\:\s*\"(?<dest>[^\"]*)
EXTRACT-amp_file_name = \"file_name\"\:\s*\"(?<file_name>[^\"]*)
EXTRACT-amp_file_path = \"file_path\"\:\s*\"(?<file_path>[^\"]*)
EXTRACT-amp_user = \"user\"\:\s?\"(?<user>[^\"]+)
EVAL-signature =
EVAL-action =
EVAL-file_hash =
BREAK_ONLY_BEFORE = ([\r\n]*)\{\"event_type\"\:
DATETIME_CONFIG =
NO_BINARY_CHECK = true
disabled = false
INDEXED_EXTRACTIONS = json
0 Karma

jplumsdaine22
Influencer

in your inputs.conf (on the forwarder) setting you need this:

[monitor://path/to/file]
INDEXED_EXTRACTIONS = json

then in props use
[cisco:amp:json]
TIMESTAMP_FIELDS = timestamp

0 Karma

danbar6
Explorer

Did you check _internal logs for timestamp extraction related errors or warnings?

0 Karma

niketn
Legend

@mkarimi17, the JSON in the sample, is that not how the data is sent? i.e. date: 2018-03-08T11:42:57+00:00 does not exist in your _raw data.

If your Timezone is set to PST using Logged in User > Account Settings in Splunk, then you should see event time as PST although the logged time in raw data is UTC.

What time do you see when you try to display the following:

<YourBaseSearch>
| table _time date_hour _raw
____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

gjanders
SplunkTrust
SplunkTrust

Are you looking for the timezone setting in the props.conf file ? As in TZ = UTC or similar?

0 Karma

niketn
Legend

@garethatiag, I have converted to comment since it seemed more like a question than answer.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

mkarimi17
Path Finder

I tried that as well, sorry forgot to include it. But yeah that doesn't work either.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Laser Bananas and Edge Hubs: Exploring Operational Technology (OT) Data Through a ...

  OT is a different environment to traditional IT and can have interesting challenges when interfacing the ...

Event Series: Mastering AI Tokenomics and Splunk Agent Observability

Beyond the Black Box: Correlating AI Performance and Tokenomics with Splunk Agent Observability   As ...

span_metrics: The OpenTelemetry-Idiomatic Way to See Inside Your Services

You open a trace in Splunk Observability Cloud and everything looks fine. One root span, order-pipeline, with ...