Splunk Search

extracting a data field

Caio_Santos
Path Finder

I have one event viewer log and I'm tryng to capture the data fields, since Splunk cannot recognize the timstamp by itself.

Here is one log sampe:

,1180580,NTBackup,**20100920230222**.000000-180,20100920230222.000000-180,Information,
spsnip01,16,ShadowCopy,SPSNIP01,,2003,2003,3,{lserver;3092;;15},Application,lserver (3092) Shadow copy 15 freeze stopped.
,1180579,ESENT,**20100920230139**.000000-180,20100920230139.000000-180,Information,
spsnip01,16,ShadowCopy,SPSNIP01,,2001,2001,3,{lserver;3092;;15},Application,lserver (3092) Shadow copy 15 freeze started.
,1180578,ESENT,**20100920230138**.000000-180,20100920230138.000000-180,Information,
spsnip01,0,,SPSNIP01,,8018,8018,3,,Application,Begin Operation
,1180577,NTBackup,**20100920230007**.000000-180,20100920230007.000000-180,Information,
spsnip01,0,,SPSNIP01,,0,0,3,{20100920_223511 - Arquivo procrelperf_ccccp02_nfe_ago10.xls processado OK},Application,20100920_223511 - Arquivo procrelperf_ccccp02_nfe_ago10.xls processado OK
,1180576,PerfReportGeneratorService,**20100920223543**.000000-180,20100920223543.000000-180,Information,
spsnip01,0,,SPSNIP01,,0,0,3,{20100920_223511 - Processando arquivo procrelperf_ccccp02_nfe_ago10.xls},Application,20100920_223511 - Processando arquivo procrelperf_ccccp02_nfe_ago10.xls
,1180575,PerfReportGeneratorService,20100920223512.000000-180,20100920223512.000000-180,Information,
spsnip01,0,,SPSNIP01,,6,-2147352570,2,{50;60},Application,Reached crypt32 threshold of 50 events and will suspend logging for 60 minutes
,1180574,crypt32,20100920220731.000000-180,20100920220731.000000-180,Warning,

How can I tell splunk how to correctly read the timestamp (bold) ? Thanks

Ok, got it done. I updated my props.conf to

[NTBackup]
DATETIME_CONFIG = CURRENT
TIME_PREFIX = (?m)^,\d+,\w+,
TIME_FORMAT = %y%m%d%H%M%S

restarted splunk, but not happened. still get splunk with just one timestamp [the timestamp when i got my data added]

I tryed as well to set the sourcetype [csv] up as CHECK_FOR_HEADER=TRUE and hoped that splunk would do its part as described on the documentation http://www.splunk.com/base/Documentation/latest/Admin/Extractfieldsfromfileheadersatindextime

I restarted the instance, but still the same. Genti, I've changed to the other values, but without success.

Tags (1)
1 Solution

Genti
Splunk Employee
Splunk Employee

If you want to extract a data field then you can do the following:
Note this is a log file that has , as delimiters.
You can then use two methods:
1 - using delims: http://www.splunk.com/base/Documentation/4.1.5/Knowledge/Createandmaintainsearch-timefieldextraction...

2 - Using a regex. (note that your timestamp is always the 4th field, so setting up a regex that grabs this should work) something like: ^[^,]*,[^,]*,[^,]*,(\w+)
Basically a props.conf that does something like this:

[yoursourcetype]  
EXTRACT-TIME = ^[^,]*,[^,]*,[^,]*,(?P<time>\w+)  

However i think you can actually get the correct timestamp by trying to modify this: http://www.splunk.com/base/Documentation/4.1.5/admin/Configuretimestamprecognition

specifically:

[yoursource or sourcetype]  
DATETIME_CONFIG = <filename relative to $SPLUNK_HOME>  
MAX_TIMESTAMP_LOOKAHEAD = <integer>   
TIME_PREFIX = <regular expression>  
TIME_FORMAT = <strptime-style format>  
TZ = <posix timezone string>  
MAX_DAYS_AGO = <integer>  
MAX_DAYS_HENCE = <integer>  

so in your case i think it should be:

TIME_PREFIX = ^[^,]*,[^,]*,[^,]*,  
TIME_FORMAT = %y%m%d%H%M%S

Try that and let me know if it works,
.gz

View solution in original post

Genti
Splunk Employee
Splunk Employee

Set DATETIME_CONFIG = CURRENT to assign the current system time to each event as it's indexed. Setting datetime_config to current is counterproductive to what you are trying to achieve.

0 Karma

Genti
Splunk Employee
Splunk Employee

If you want to extract a data field then you can do the following:
Note this is a log file that has , as delimiters.
You can then use two methods:
1 - using delims: http://www.splunk.com/base/Documentation/4.1.5/Knowledge/Createandmaintainsearch-timefieldextraction...

2 - Using a regex. (note that your timestamp is always the 4th field, so setting up a regex that grabs this should work) something like: ^[^,]*,[^,]*,[^,]*,(\w+)
Basically a props.conf that does something like this:

[yoursourcetype]  
EXTRACT-TIME = ^[^,]*,[^,]*,[^,]*,(?P<time>\w+)  

However i think you can actually get the correct timestamp by trying to modify this: http://www.splunk.com/base/Documentation/4.1.5/admin/Configuretimestamprecognition

specifically:

[yoursource or sourcetype]  
DATETIME_CONFIG = <filename relative to $SPLUNK_HOME>  
MAX_TIMESTAMP_LOOKAHEAD = <integer>   
TIME_PREFIX = <regular expression>  
TIME_FORMAT = <strptime-style format>  
TZ = <posix timezone string>  
MAX_DAYS_AGO = <integer>  
MAX_DAYS_HENCE = <integer>  

so in your case i think it should be:

TIME_PREFIX = ^[^,]*,[^,]*,[^,]*,  
TIME_FORMAT = %y%m%d%H%M%S

Try that and let me know if it works,
.gz

View solution in original post

Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.