topic Re: Can I change _time based on time within data that is ingested in Splunk Search

Can I change _time based on time within data that is ingested

janesh22 — Wed, 27 Jan 2021 03:57:34 GMT

Hi ,

I have a report that is ingested in splunk. Due to the report format not correctly ingested by splunk, I had done some preprocessing of the report and named reportfile.rep and picked up by splunk every 15 mins. This report is delayed almost 40 minutes(as the processing and transferring of data takes time) , so the time stamp indexed by splunk is around 40 minutes delayed from the report. example:

sample file:

reportfile.rep

ReportID=a004_012721.1400,Queue=xxx,AgentList=xxxx

ReportID=a004_012721.1400,Queue=xxx

_time = 2021-01-27T14:40:04.000+11:00

So report was for 14:00 however _time is 14:40.

Is there anyway I can overwrite _time value to be picked up from the report file.

I had seen some examples in splunk answers using transforms.conf and props.conf. However this is based on using the actual file name and not the content inside the file.

Re: Can I change _time based on time within data that is ingested

scelikok — Wed, 27 Jan 2021 04:32:12 GMT

Hi @janesh22,

You can use TIME_PREFIX and TIME_FORMAT like below;

[report_sourcetype] TIME_PREFIX = ReportID\=\w\d{3}_ TIME_FORMAT = %m%d%y.%H%M

Re: Can I change _time based on time within data that is ingested

janesh22 — Wed, 27 Jan 2021 05:18:32 GMT

Thanks @scelikok

So do I just add these in props.conf and restart splunk to enable it?

Re: Can I change _time based on time within data that is ingested

scelikok — Wed, 27 Jan 2021 05:26:58 GMT

You're welcome @janesh22,

If you are running standalone Splunk it should be enough. If you are getting these reports using Universal Formarder, you should put this to indexers.

Please do not forget to change stanza name "report_sourcetype" with your real sourcetype.

Re: Can I change _time based on time within data that is ingested

janesh22 — Thu, 28 Jan 2021 01:11:25 GMT

Hi @scelikok

I had added the below to proc.conf with the sourcetype name, however it still does not seem to be picking up the time . I had tried restarting the splunk as well.

Does this show up in any logs , if it is actually checking the conf file ?

Regards

Janesh