How to configure Splunk to use a timestamp field in my sample log as _time?

[rid1]

Hi,

I'm new in Splunk, hope you can guide step by step please.

How do I map or link a timestamp field (eg. timestamp_mrt) into _time so I can search on it?
Right now my timestamp is none.

Here the log that I loaded into Splunk

{
    "received_bytes": "28942",
    "srcip": "***************",
    "src_port": "42012",
    "dstport": "443",
    "action": "allow",
    "type": "checkpoint_fw",
    "dst_port": "443",
    "log_sequence_num": "0",
    "LastUpdateTime": "1512314958",
    "host": "**************",
    "action": "allow",
    "dstip": "**************",
    "__policy_id_tag": "product=VPN-1 & FireWall-1[db_tag={10F51296-63EA-884A-BF05-579D4499EA21}",
    "app_id": "60519733",
    "protocol": "tcp",
    "app_rule_name": " ",
    "timestamp_mrt": "2017-12-03T15:30:36.208Z",
    "is_first_for_luuid": "0",
    "reporting_host": "****************",
    "Suppressed logs": "19",
    "browse_time": "0:00:00",
    "host": "***************",
    "dst_ip": "**************",
    "port": 51354,
    "i/f_name": "eth1-01",
    "proto": "tcp",
    "app_risk": "1",
    "src_ip": "*****************",
    "app_rule_id": "{B9494762-1D04-444B-A1F5-373B2DEC6CEE}",
    "app_category": "Network Protocols",
    "log_version": "1",
    "reportingHost": "*****************",
    "appi_name": "HTTP/2 over TLS",
    "rule": " ",
    "app_sig_id": "60519733:10",
    "cp_fwProductName": "Application Control",
    "typeNo": "13",
    "sent_bytes": "24332",
    "cp_date": "03Dec2017 14:50:17",
    "app_properties": "Very Low Risk, Network Protocols, Communication Standard",
    "log_type": "log",
    "has_accounting": "0",
    "cp_hostip": "**************",
    "@version": "1",
    "logId": "-1",
    "matched_category": "Network Protocols",
    "proxy_src_ip": "172.26.2.10",
    "product": "Application Control",
    "i/f_dir": "outbound",
    "origin_sic_name": "CN=fw1,O=fw..vtvb5n",
    "@timestamp": "2017-12-03T15:30:36.208Z",
    "bytes": "53274",
    "srcport": "42012",
    "app_desc": "HTTP/2 provides an optimized transport for HTTP semantics. HTTP/2 supports all of the core features of HTTP/1.1, but aims to be more efficient in several ways. HTTP/2 over TLS allows HTTP/2 connections over a secured TLS connection. Supported from: R75.40."
  }

thank you!!

[mayurr98 · Accepted solution]

put this in props.conf

TIME_PREFIX =  timestamp_mrt\"\:\s\"
TIME_FORMAT = %Y-%d-%mT%H:%M:%S.%3QZ

[rid1]

I tried both method but I still can't search for the log with
- search is *
-condition between 2 dec 2017 to 4th dec 2017

[rid1]

perfect! as @deepashri_123 and you recommended to apply to new events.

[mayurr98]

Are you able to see _time and timestamp_mrt same in the raw logs after doing above configuration.

For your info, you need to restart Splunk server after doing this configuration.
Also, this configuration will apply to latest events only i.e. from the time you restart the Splunk server it will not apply on historical events so check real-time latest events.

[rid1]

Thank you guys

[deepashri_123]

Hey rid1,

You can create a custom sourcetype and in props.conf you can add
TIME_PREFIX = \"timestamp_mrt\":
You also need to add TIME_FORMAT in your props.
Refer this link below:
https://docs.splunk.com/Documentation/Splunk/7.0.2/Admin/Propsconf
http://docs.splunk.com/Documentation/Splunk/7.0.2/Data/Createsourcetypes

Hope this helps!!

[rid1]

still giving my "timestamp = none"
thus I tried to find the log with condition of search
2nd dec 2017 to 4th dec 2017, still give me nothing.

[deepashri_123]

hey rid1,

I think you will have to re-index your data to see the config changes as the changes wont be implemented on data that is already indexed. It will be added only to the new data that is indexed.

[rid1]

new sourcetype created, and below were added, service restarted.

TIME_PREFIX = \"timestamp_mrt\":
TIME_FORMAT = %m/%d/%Y

I still can't query based on the time which is 3rd december 2017.

here is some result after prefix were added:

@timestamp
2017-12-03T15:30:36.208Z

timestamp
none

timestamp_mrt
2017-12-03T15:30:36.208Z

[mayurr98]

TIME_PREFIX = \"timestamp_mrt\"\: this won't work as it should be TIME_PREFIX = \"timestamp_mrt\"\:\s\"