topic Re: timestamp extraction issue in Getting Data In

timestamp extraction issue

abhinav_bel — Sat, 04 Jul 2020 03:47:40 GMT

Hi ,

My timestamp in data looks like: 2020-07-02T18:00:18+02:00 with name log_modified_date which i want to be extracted

i have written below props.conf:

[_json]
INDEXED_EXTRACTIONS = json
KV_MODE = none
AUTO_KV_JSON = false
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIMESTAMP_FIELDS = last_modified_date
TIME_FORMAT = %Y-%m-%dT%H:%M:%S+%2N:%2N
MAX_TIMESTAMP_LOOKAHEAD = 25

and getting time extracted as :

7/2/20
6:00:18.020 PM

but I want the time field extracted in same way as in data with + value as well like:

7/2/20
6:00:18+02:00

Please let me know what i am doing wrong as i am not getting expected output with + value.

Note： this +02：00 value is fixed with every timestamp in data .

Here’s my sample log data：

{"_timestamp":"2020-07-02 18:00:46","_ver":"2","asset_name":"","assigned_group":"Troubleshooting - Tier 2","assignee":"Buhle Mahlaba","ci":"","cause":"","city":"","client_type":"","closed_date":"","closure_source":"","company":"MTN BUSINESS","contact_phone":"","contact_site":"","country":"","created_from_template":"","customer_phone":"###","customer_site":"INTERNET SOLUTIONS(PTY) LTD","debtor_code":"MTN000","direct_contact_city":"","direct_contact_company":"","direct_contact_corporate_id":"","direct_contact_country":"","direct_contact_country_code":"","direct_contact_department":"","direct_contact_desk_location":"","direct_contact_extension":"","direct_contact_first_name":"","direct_contact_internet_email":"","direct_contact_last_name":"","direct_contact_local_number":"","direct_contact_location_details":"","direct_contact_middle_initial":"","direct_contact_organization":"","direct_contact_region":"","direct_contact_site_group":"","direct_contact_state_province":"","direct_contact_street":"","direct_contact_time_zone":"","direct_contact_zip_postal_code":"","first_name":"Melvern","impact":"2-Significant\/Large","incident_id":"MTNB00001289400","incident_type":"User Service Restoration","last_acknowledged_date":"","last_modified_by":"412877","last_modified_date":"2020-07-02T18:00:44+02:00","last_name":"Banoo","last_resolved_date":"","middle_name":"","notes":"HI Team\n\nThe mentioned link is down ,Please investigate and advise.\n\n\nRP\/0\/RSP0\/CPU0:mi-za-bry-mspe4#sho log | inc BVI906\nRP\/0\/RSP0\/CPU0:Jul 2 14:43:49.894 SAST: mpls_ldp[1204]: %ROUTING-LDP-5-HELLO_ADJ_CHANGE : VRF 'default' (0x60000000), Link hello adja...","operational_categorization_tier_1":"TES_Link","operational_categorization_tier_2":"Microwave PTP","operational_categorization_tier_3":"Link Down","owner_group":"General Support","priority":"Critical","product_categorization_tier_1":"TES_Managed Networks","product_categorization_tier_2":"Access Service","product_categorization_tier_3":"Cloud Connect","product_name":"","region":"","reported_date":"2020-07-02T16:36:04+02:00","reported_source":"Email","resolution":"","resolution_categorization_tier_1":"","resolution_categorization_tier_2":"","resolution_categorization_tier_3":"","resolution_product_categorization_tier_1":"","resolution_product_categorization_tier_2":"","resolution_product_categorization_tier_3":"","responded_date":"2020-07-02T18:00:43+02:00","slm_real_time_status":"Within the Service Target","satisfaction_rating":"","service_manager":"","service_request_id":"","site_group":"","state_province":"","status":"In Progress","status_reason_hidden":"","street":"","submit_date":"2020-07-02T16:36:04+02:00","submitter":"AR_ESCALATOR","summary":"INC000147465| me-za-gp80-hoedspru-bry-1 | | E2379","time_zone":"","urgency":"1-Critical","vendor_group":"","vendor_name":"","vendor_ticket_number":"","zip_postal_code":""}

Re: timestamp extraction issue

to4kawa — Sat, 04 Jul 2020 07:42:04 GMT

Re: timestamp extraction issue

abhinav_bel — Sat, 04 Jul 2020 04:59:05 GMT

Hi ,

Thanks for your quick response but it is also not giving output as expected.

using transforms.conf , you are adding +2:00 value in time and thats why when running it is giving output:

7/2/20
6:02:44.000 PM

But as I told I want :

7/2/20
6:00:44+02:00 PM

Exactly the same way as in sample log last_modified_date has value.

Please try to understand I don't want to add that value in time, i want in same format as mentioned .

Re: timestamp extraction issue

to4kawa — Sat, 04 Jul 2020 07:42:21 GMT

Re: timestamp extraction issue

abhinav_bel — Sat, 04 Jul 2020 05:20:52 GMT

Ahh that might work.

Could you please elaborate how and where to change in user preference and timezone(+02:00) ?

Thanks

Re: timestamp extraction issue

to4kawa — Sat, 04 Jul 2020 07:42:39 GMT

Re: timestamp extraction issue

abhinav_bel — Sat, 04 Jul 2020 06:28:09 GMT

Sorry that doesn't work I changed user preference and time zone (+2) but that too will change my time according to that time zone.

This is not what I want which i told several times.

I want +2:00 value included in my _time field that is same as log_modified_date field.

Hope now you understand what I want .

There is definitely some way through time format or transforms we can achieve this :

7/2/20
6:00:44+02:00 PM

Can anyone help me on this?

Re: timestamp extraction issue

abhinav_bel — Tue, 07 Jul 2020 12:49:40 GMT

Hi ,

My timestamp in data looks like: 2020-07-02T18:00:18+02:00 with name log_modified_date which i want to be extracted

i have written below props.conf:

and getting time extracted as :

7/2/20
6:00:18.020 PM

but I want the time field extracted in same way as in data with + value as well like:

7/2/20
6:00:18+02:00

Please let me know what i am doing wrong as i am not getting expected output with + value.

Note： this +02：00 value is fixed with every timestamp in data .

Here’s my sample log data：