topic Re: How to configure Splunk to use a timestamp field in my sample log as _time? in Splunk Search

How to configure Splunk to use a timestamp field in my sample log as _time?

rid1 — Tue, 27 Feb 2018 01:48:52 GMT

Hi,

I'm new in Splunk, hope you can guide step by step please.

How do I map or link a timestamp field (eg. timestamp_mrt) into _time so I can search on it?
Right now my timestamp is none.

Here the log that I loaded into Splunk

{
    "received_bytes": "28942",
    "srcip": "***************",
    "src_port": "42012",
    "dstport": "443",
    "action": "allow",
    "type": "checkpoint_fw",
    "dst_port": "443",
    "log_sequence_num": "0",
    "LastUpdateTime": "1512314958",
    "host": "**************",
    "action": "allow",
    "dstip": "**************",
    "__policy_id_tag": "product=VPN-1 & FireWall-1[db_tag={10F51296-63EA-884A-BF05-579D4499EA21}",
    "app_id": "60519733",
    "protocol": "tcp",
    "app_rule_name": " ",
    "timestamp_mrt": "2017-12-03T15:30:36.208Z",
    "is_first_for_luuid": "0",
    "reporting_host": "****************",
    "Suppressed logs": "19",
    "browse_time": "0:00:00",
    "host": "***************",
    "dst_ip": "**************",
    "port": 51354,
    "i/f_name": "eth1-01",
    "proto": "tcp",
    "app_risk": "1",
    "src_ip": "*****************",
    "app_rule_id": "{B9494762-1D04-444B-A1F5-373B2DEC6CEE}",
    "app_category": "Network Protocols",
    "log_version": "1",
    "reportingHost": "*****************",
    "appi_name": "HTTP/2 over TLS",
    "rule": " ",
    "app_sig_id": "60519733:10",
    "cp_fwProductName": "Application Control",
    "typeNo": "13",
    "sent_bytes": "24332",
    "cp_date": "03Dec2017 14:50:17",
    "app_properties": "Very Low Risk, Network Protocols, Communication Standard",
    "log_type": "log",
    "has_accounting": "0",
    "cp_hostip": "**************",
    "@version": "1",
    "logId": "-1",
    "matched_category": "Network Protocols",
    "proxy_src_ip": "172.26.2.10",
    "product": "Application Control",
    "i/f_dir": "outbound",
    "origin_sic_name": "CN=fw1,O=fw..vtvb5n",
    "@timestamp": "2017-12-03T15:30:36.208Z",
    "bytes": "53274",
    "srcport": "42012",
    "app_desc": "HTTP/2 provides an optimized transport for HTTP semantics. HTTP/2 supports all of the core features of HTTP/1.1, but aims to be more efficient in several ways. HTTP/2 over TLS allows HTTP/2 connections over a secured TLS connection. Supported from: R75.40."
  }

thank you!!

Re: How to configure Splunk to use a timestamp field in my sample log as _time?

deepashri_123 — Tue, 29 Sep 2020 18:27:46 GMT

Hey rid1,

You can create a custom sourcetype and in props.conf you can add
TIME_PREFIX = \"timestamp_mrt\":
You also need to add TIME_FORMAT in your props.
Refer this link below:
https://docs.splunk.com/Documentation/Splunk/7.0.2/Admin/Propsconf
http://docs.splunk.com/Documentation/Splunk/7.0.2/Data/Createsourcetypes

Hope this helps!!

Re: How to configure Splunk to use a timestamp field in my sample log as _time?

mayurr98 — Tue, 27 Feb 2018 05:54:43 GMT

put this in props.conf

TIME_PREFIX =  timestamp_mrt\"\:\s\"
TIME_FORMAT = %Y-%d-%mT%H:%M:%S.%3QZ

Re: How to configure Splunk to use a timestamp field in my sample log as _time?

rid1 — Tue, 29 Sep 2020 18:16:07 GMT

new sourcetype created, and below were added, service restarted.

TIME_PREFIX = \"timestamp_mrt\":
TIME_FORMAT = %m/%d/%Y

I still can't query based on the time which is 3rd december 2017.

here is some result after prefix were added:

@timestamp
2017-12-03T15:30:36.208Z

timestamp
none

timestamp_mrt
2017-12-03T15:30:36.208Z

Re: How to configure Splunk to use a timestamp field in my sample log as _time?

rid1 — Tue, 27 Feb 2018 06:00:59 GMT

I tried both method but I still can't search for the log with
- search is *
-condition between 2 dec 2017 to 4th dec 2017

Re: How to configure Splunk to use a timestamp field in my sample log as _time?

rid1 — Tue, 27 Feb 2018 06:02:11 GMT

still giving my "timestamp = none"
thus I tried to find the log with condition of search
2nd dec 2017 to 4th dec 2017, still give me nothing.

Re: How to configure Splunk to use a timestamp field in my sample log as _time?

mayurr98 — Tue, 27 Feb 2018 06:06:07 GMT

Are you able to see _time and timestamp_mrt same in the raw logs after doing above configuration.

For your info, you need to restart Splunk server after doing this configuration.
Also, this configuration will apply to latest events only i.e. from the time you restart the Splunk server it will not apply on historical events so check real-time latest events.

Re: How to configure Splunk to use a timestamp field in my sample log as _time?

deepashri_123 — Tue, 27 Feb 2018 06:07:50 GMT

hey rid1,

I think you will have to re-index your data to see the config changes as the changes wont be implemented on data that is already indexed. It will be added only to the new data that is indexed.

Re: How to configure Splunk to use a timestamp field in my sample log as _time?

rid1 — Tue, 27 Feb 2018 07:51:05 GMT

perfect! as @deepashri_123 and you recommended to apply to new events.

Re: How to configure Splunk to use a timestamp field in my sample log as _time?

rid1 — Tue, 27 Feb 2018 07:51:31 GMT

Thank you guys

Re: How to configure Splunk to use a timestamp field in my sample log as _time?

mayurr98 — Tue, 27 Feb 2018 13:46:49 GMT

TIME_PREFIX = \"timestamp_mrt\"\: this won't work as it should be TIME_PREFIX = \"timestamp_mrt\"\:\s\"