When indexing don't create an event every time EPOCH date is shown, but still index the data

[j666gak]

Hi,

I'm having a bit of a headache. I am trying to index an XML file however I want the event date to be the date the file is updated, and not each EPOCH date listed in the file. Although I do still want to index this data.

The second thing that I want to do is is convert the EPOCH date in to a human readable format.

Example of log file

domain created="1364516883">url.com /domain>

domain created="1364516882">url.net /domain>

domain created="1364516882">url.info /domain>

I would appreciate any help anybody can give me.

Thanks

[gkanapathy]

You can do this force this by creating a datetime.xml file containing only:

<timePatterns />
<dataPatterns />

and then indexing with

DATETIME_CONFIG = /etc/apps/myapp/local/mydatetime.xml

or whatever the appropriate relative path to your file is. This will cause date parsing from the data and from the file name to fail (because no patterns exist that can be matched), until the remaining option is to use the modification date of the input file. Instead of trying to modify the epoch date when the data is indexed, you should just convert it at search time when viewing the data, e.g., using the | convert search command or strptime() eval function.

[Ayn]

Use DATETIME_CONFIG = CURRENT in props.conf.

http://docs.splunk.com/Documentation/Splunk/latest/admin/Propsconf

As for converting epoch to human readable in the actual raw event, that's not possible to do via an index-time transform at least. If it's really important to you, you could consider switching to a script-based input and put that kind of transforming logic into the script instead.

[Ayn]

Yes. The assumption would be that Splunk will pick events up right as the file is updated, so index time will be roughly the same as file modification time.

[gkanapathy]

This will set the timestamp to the time the file is indexed, rather than the modification time of the file.