I have basic 4 types of files. under
C:\Program Files\Splunk\etc\apps\my logs\home_logs\KLZ\host1\ abck_KLZ_CPU_110213.csv abck01_KLZ_Disk_110213.csv abck01_KLZ_Network_110213.csv abck01_KLZ_Swap_Rate_110213.csv ...... C:\Program Files\Splunk\etc\apps\my logs\home_logs\KLZ\host2\ defg_KLZ_CPU_110213.csv defg01_KLZ_Disk_110213.csv defg01_KLZ_Network_110213.csv defg01_KLZ_Swap_Rate_110213.csv ...... ..... .....
There are lots of host and its corrosponding files.
I wanted 4 types of sourcetype ie.
network, cpu, swap-rate and
and host name should be
for this my conf files are as below:
[monitor:C:\Program Files\Splunk\etc\apps\my logs\home_logs\..\..\*] disabled = false index = my_indx host_segment = 8
CHARSET=SHIFT-JIS [source::...Disk...] sourcetype = disk [source::...CPU...] sourcetype = cpu [source::...Network...] sourcetype = network [source::...Swap_Rate...] sourcetype = swap-rate
The result using above confs. Only 1 directory getting indexed. and for other directories only 1 file (same sourcetype) getting indexed. Other files are not indexed.
Hope you understand my problem. Your help will be appreciated.
Thanks in advance
I think the application name is wrong.
Application name can only contain the following characters: a-zA-Z0-9_-
Please check your splunkd.log (C:\Program Files\Splunk\var\log\splunk\splunkd.log)
If there are error messages such as bellow, please add the setting in configuration: "
crcSalt = <SOURCE>" In inputs.conf
02-28-2011 15:00:00.000 ERROR TailingProcessor - Ignoring path due to: File will not be read, seekptr checksum did not match (file=XXXXX). Last time we saw this initcrc, filename was different. You may wish to use a CRC salt on this source. Consult the documentation or file a support case online at http://www.splunk.com/page/submit_issue for more info.
*Note: For more information, please see: http://www.splunk.com/base/Documentation/latest/Admin/Monitorfilesanddirectories#Monitor_syntax_and_...
Your posted config should work, except for the monitor. If I understand what you're trying to accomplish, I would expect it to look like this:
[monitor://C:\Program Files\Splunk\etc\apps\my logs\home_logs\] disabled = false recursive = true host_segment = 8 index = my_indx crcSalt = <SOURCE>
EDIT: added crcSalt to resolve 'too small' error.
This will index everything in the home_logs folder and all sub-folders. Recursive is the default, but declaring it will help ensure there's not an override somewhere else.
I am getting the error while indexing:
02-25-2011 19:41:58.030 ERROR TailingProcessor - Ignoring path due to: File will not be read, is too small to match seekptr checksum (file=C:\EDN\test01\kednwbs01KLZDisk110213.csv). Last time we saw this initcrc, filename was different. You may wish to use a CRC salt on this source. Consult the documentation or file a support case online at http://www.splunk.com/page/submitissue for more info.
What do you think of this approach:
inputs.conf ----------- [monitor:C:\Program Files\Splunk\etc\apps\my logs\home_logs\...] disabled = false index = my_indx props.conf ---------- [source::C:\Program Files\Splunk\etc\apps\my logs\home_logs\...] TRANSFORMS-checkpath = sourcetype-transform, host-transform transforms.conf --------------- [sourcetype-transform] SOURCE_KEY = MetaData:Source # defg _ KLZ _ CPU _110213.csv REGEX= .*/[^_]+_[^_]+_([^_\d]+).*\.csv # just for the case the regex doesn't match #DEFAULT_VALUE = sourcetype::default-sourcetype DEST_KEY = MetaData:Sourcetype FORMAT = sourcetype::$1 [host-transform] SOURCE_KEY = MetaData:Source # defg _ KLZ _ CPU _110213.csv REGEX= .*/([^_])+_[^_]+_[^_\d]+.*\.csv # just for the case the regex doesn't match #DEFAULT_VALUE = host::default-host DEST_KEY = MetaData:Host FORMAT = sourcetype::$1
obs: I did not test this configuration and maybe there are small errors in it. But I want to show that you can examine the path (MetaData:Source) of your input to create or modify basic fields (source, sourcetype, host) at index time.