Remove additional timestamp from the logs

zaan — Wed, 01 Jul 2020 16:04:00 GMT

Hi All

i have onboarded linux logs from S3--> Splunk . I found additional timestamp is getting attached to the events. Can you please help me in removing the additional timestamp. Below is the expected log format.

Before,

2020-07-01T10:59:58Z messages {"message":"Jun 1 10:59:58 stg-coinbrh: [get_meta] Trying to get http://10.4.3.1/latest/meta-data/network/interfaces/macs/06:c3:45:12:56:12/subnet-ipv4-cidr-block"}
2020-07-01T10:59:58Z messages {"message":"Jun 4 10:59:58 stg-mbcoln: [rewrite_aliases] Rewriting aliases of eth0"}

After,

Jun 1 10:59:58 stg-coinbrh: [get_meta] Trying to get http://10.4.3.1/latest/meta-data/network/interfaces/macs/06:c3:45:12:56:12/subnet-ipv4-cidr-block

Jun 4 10:59:58 stg-mbcoln: [rewrite_aliases] Rewriting aliases of eth0

Please help me in defining exact props and transforms settings to achieve this.

Thanks in advance

Re: Remove additional timestamp from the logs

richgalloway — Wed, 01 Jul 2020 17:13:06 GMT

Depending on how you're getting the data from S3 to Splunk there may be other, better answers, but using SEDCMD should work. Add this line to the props.conf file for the sourcetype.

SEDCMD-unjson = s/\{"message":"(.*)"}/\1/g

topic Re: Remove additional timestamp from the logs in Splunk Search

Remove additional timestamp from the logs

Re: Remove additional timestamp from the logs