Dear All,
I have basic 4 types of files. under
C:\Program Files\Splunk\etc\apps\my logs\home_logs\KLZ\host1\
abck_KLZ_CPU_110213.csv
abck01_KLZ_Disk_110213.csv
abck01_KLZ_Network_110213.csv
abck01_KLZ_Swap_Rate_110213.csv
......
C:\Program Files\Splunk\etc\apps\my logs\home_logs\KLZ\host2\
defg_KLZ_CPU_110213.csv
defg01_KLZ_Disk_110213.csv
defg01_KLZ_Network_110213.csv
defg01_KLZ_Swap_Rate_110213.csv
......
.....
.....
There are lots of host and its corrosponding files.
I wanted 4 types of sourcetype ie. network, cpu, swap-rate
and disk
.
and host name should be host1,host2,....
for this my conf files are as below:
inputs.conf:
[monitor:C:\Program Files\Splunk\etc\apps\my logs\home_logs\..\..\*]
disabled = false
index = my_indx
host_segment = 8
props.conf:
CHARSET=SHIFT-JIS
[source::...Disk...]
sourcetype = disk
[source::...CPU...]
sourcetype = cpu
[source::...Network...]
sourcetype = network
[source::...Swap_Rate...]
sourcetype = swap-rate
The result using above confs. Only 1 directory getting indexed. and for other directories only 1 file (same sourcetype) getting indexed. Other files are not indexed.
Hope you understand my problem. Your help will be appreciated.
Thanks in advance
Hi msona,
I think the application name is wrong.
Application name can only contain the following characters: a-zA-Z0-9_-
Please check your splunkd.log (C:\Program Files\Splunk\var\log\splunk\splunkd.log)
If there are error messages such as bellow, please add the setting in configuration: "crcSalt = <SOURCE>
" In inputs.conf
02-28-2011 15:00:00.000 ERROR TailingProcessor - Ignoring path due to: File will not be read, seekptr checksum did not match (file=XXXXX). Last time we saw this initcrc, filename was different. You may wish to use a CRC salt on this source. Consult the documentation or file a support case online at http://www.splunk.com/page/submit_issue for more info.
*Note: For more information, please see: http://www.splunk.com/base/Documentation/latest/Admin/Monitorfilesanddirectories#Monitor_syntax_and_...
Your posted config should work, except for the monitor. If I understand what you're trying to accomplish, I would expect it to look like this:
[monitor://C:\Program Files\Splunk\etc\apps\my logs\home_logs\]
disabled = false
recursive = true
host_segment = 8
index = my_indx
crcSalt = <SOURCE>
EDIT: added crcSalt to resolve 'too small' error.
This will index everything in the home_logs folder and all sub-folders. Recursive is the default, but declaring it will help ensure there's not an override somewhere else.
I'll edit the answer to show how to add the CRC salt. Use
I am getting the error while indexing:
02-25-2011 19:41:58.030 ERROR TailingProcessor - Ignoring path due to: File will not be read, is too small to match seekptr checksum (file=C:\EDN\test01\kednwbs01_KLZ_Disk_110213.csv). Last time we saw this initcrc, filename was different. You may wish to use a CRC salt on this source. Consult the documentation or file a support case online at http://www.splunk.com/page/submit_issue for more info.
Hi msona,
What do you think of this approach:
inputs.conf
-----------
[monitor:C:\Program Files\Splunk\etc\apps\my logs\home_logs\...]
disabled = false
index = my_indx
props.conf
----------
[source::C:\Program Files\Splunk\etc\apps\my logs\home_logs\...]
TRANSFORMS-checkpath = sourcetype-transform, host-transform
transforms.conf
---------------
[sourcetype-transform]
SOURCE_KEY = MetaData:Source
# defg _ KLZ _ CPU _110213.csv
REGEX= .*/[^_]+_[^_]+_([^_\d]+).*\.csv
# just for the case the regex doesn't match
#DEFAULT_VALUE = sourcetype::default-sourcetype
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::$1
[host-transform]
SOURCE_KEY = MetaData:Source
# defg _ KLZ _ CPU _110213.csv
REGEX= .*/([^_])+_[^_]+_[^_\d]+.*\.csv
# just for the case the regex doesn't match
#DEFAULT_VALUE = host::default-host
DEST_KEY = MetaData:Host
FORMAT = sourcetype::$1
obs: I did not test this configuration and maybe there are small errors in it. But I want to show that you can examine the path (MetaData:Source) of your input to create or modify basic fields (source, sourcetype, host) at index time.
Can Anybody Help me Please ????
for the host name I have added
host_segment = 8 in the input.conf
Hi meno,
Thanks for the answer.
I checked but its not working :(. Its taking default host and source type.
I want the host as directory name and sourcetype as some part of file name.