Getting Data In

what could be the reason for some splunk sessions observed to be in the DDMMYYYY format ( ideally it is in MMDDYYYY format)?

pavanae
Builder

Default date in the Splunk session is observed to be in the DDMMYYYY format ( ideally it is in MMDDYYYY format)

Due to this Splunk session shows "No results" for these logs

Some Splunk sessions do not show any date, causing us to manually insert the date in the search query

Even when the date format was manually changed to MMDDYYYY format, some Splunk sessions show results whereas others do not

1 Solution

somesoni2
Revered Legend

Splunk doesn't change the format of the log file that you see in the log data. It'll extract the timestamp, as field _time based on the timestamp recognition rules setup on the sourcetype. Seems like your logs files are writing events with different timestamp format (dd/MM/YYYY...), which shouldn't matter as such if you've configure correct timestamp recognition rules, but if you want it to be updated, it should be updated at the source of the log file. (this looks like Apache logs, contact the data owner to get the default log format updated to your expected time format.

View solution in original post

somesoni2
Revered Legend

Splunk doesn't change the format of the log file that you see in the log data. It'll extract the timestamp, as field _time based on the timestamp recognition rules setup on the sourcetype. Seems like your logs files are writing events with different timestamp format (dd/MM/YYYY...), which shouldn't matter as such if you've configure correct timestamp recognition rules, but if you want it to be updated, it should be updated at the source of the log file. (this looks like Apache logs, contact the data owner to get the default log format updated to your expected time format.

pavanae
Builder

Thank you. And is there any way on the Splunk side to change the format from dd/MM/YYYY and display it as MM/dd/YYYY.

I heard that it can be done by adding the time_format stanza with some regex in the props.conf

0 Karma

somesoni2
Revered Legend

As I mentioned earlier, the time_format stanza is the timestamp recognition setting which recognizes the time from the raw data and set value for _time field. It doesn't update the actual raw data. This link should give you everything you need to learn about that.
http://docs.splunk.com/Documentation/Splunk/6.4.3/Data/Configuretimestamprecognition

If you're looking for updating the raw data, (not preferred) you can SEDCMD to update the raw events. Here is the details about that.
http://docs.splunk.com/Documentation/Splunk/6.4.3/Data/Anonymizedata#Anonymize_data_through_a_sed_sc...

0 Karma

jkat54
SplunkTrust
SplunkTrust

@pavanae, based on your props.conf tag, i believe you're ingesting custom data. If this is true, can you provide us with sample data and your props.conf settings for this data source?

0 Karma

pavanae
Builder

The events with date issue were as follows :-

169.36.96.254 - G302250 [31/Aug/2016:22:41:17 -0400] "GET /x/x/x/x/x/ epaxyxyxyx=ii8By25Kcge7si9zEpczZ9R4FSCPIFEw1CxfOGDfPfNraplusas8oauYhBAraequalsraequals&makesearchcallout=false&width=1200&height=300&captureframesize=false&layoutname=EPA&epausername=&QueryText=Compmon%2FMAS%20 HTTP/1.1" 200 767

But it should be as bellow

08/31/2016 12:15:09 PM
LogName=Security
SourceName=Microsoft Windows security auditing.
EventCode=4770

What I'm looking is the format of the date should be Month/Date/Year like 08/31/2016 and not in the format Date/Month/Year like 31/Aug/2016.

Also is there any possibility to make the date to be displayed in the beginning instead of middle of the event.

The following were the props.conf in 3 different directories:-

1) Splunk_Home/etc/apps/learned/local/props.conf
[splunk-config-too_small]
PREFIX_SOURCETYPE = True
SHOULD_LINEMERGE = False
is_valid = True
maxDist = 9999

[first_install-too_small]
PREFIX_SOURCETYPE = True
SHOULD_LINEMERGE = False
is_valid = True
maxDist = 9999

[splunk-config-2]
MAX_TIMESTAMP_LOOKAHEAD = 40
SHOULD_LINEMERGE = False
is_valid = True

2) Splunk_Home/etc/apps/search/local/props.conf
[splunkd]
EXTRACT-fields = (?i)^(?:[^ ]* ){2}(?:[+-]\d+ )?(?P[^ ]*)\s+(?P[^ ]+) - (?P.+)

[splunk_web_service]
EXTRACT-useragent = userAgent=(?P[^ (]+

3) Splunk_Home/etc/apps/splunkforwarder/local/props.conf

stanza that matches every string, using a priority over 100
enables us to override even literal matches. So here we disable:
(1) header line processing
[(::)?...]
CHECK_FOR_HEADER = false
priority = 10001

0 Karma

somesoni2
Revered Legend

Could you be more specific when you say "default date in Splunk Session"? Are you talking about the format of _time? May be some screenshots for issue that you're facing?

Also, check if you're browsing en-US locale URL (check the Splunk URL).

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...