Hi All, I am monitoring files that land in the same directory that I wish to be considered as different source types. The way I want to distinguish them is with their names. There will be three different source types and they will be csv files. The naming conventions will be time_*.csv, pulse_*.csv, and flow_*.csv . I actually have this working using the following in inputs.conf: [monitor://C:\tpg\leamcsv\dualgamma_logs\...\pulse_*.csv] sourcetype = DGC_PULSE index=main host_segment = 4 crcSalt = <SOURCE> [monitor://C:\tpg\leamcsv\dualgamma_logs\...\flow_*.csv] sourcetype = DGC_FLOW index=main host_segment = 4 crcSalt = <SOURCE> [monitor://C:\tpg\leamcsv\dualgamma_logs\...\time_*.csv] sourcetype = DGC_TIME index=main host_segment = 4 crcSalt = <SOURCE> This works exactly as I want. The use of crcSalt turns out to be necessary as many of the files have meta information that is identical and this forces the indexer to consider them all. As I said, the above works fine as long as the files to be monitored are landed as .csv files. My requirements have changed and I will now be landing *.zip files containing the desired .csv files. It is not clear to me why, but splunk is not indexing the zip files using the above configuration. Everything I read would seem to indicate that it should index the zip files. Perhaps the monitor stanza is excluding the zip files - I haven't been able to figure that one out. I can say that if the monitor stanza is left open( [monitor://C:\tpg\leamcsv\dualgamma_logs\...\] ), it will index the contents of the zip files, but that leaves me unable to distingush the different sourcetypes(at least not in the way that I was doing). After doing some research I read that attempting to index multiple sourcetypes from a common directory could lead to inconsistent results(I dont have that link handy at the moment). At any rate, the suggestion was to use a more open qualification as I mentioned in the previous paragraph and assign the sourcetype on a per event basis or in props.conf. I chose to do this in props.conf. I am using the following configuration: inputs.conf: [monitor://C:\tpg\leamcsv\dualgamma_logs\...\] index=main host_segment = 4 crcSalt = <SOURCE> props.conf [source::...\pulse_*\.csv] sourcetype=DGC_PULSE [source::...\flow_*\.csv] sourcetype=DGC_FLOW [source::...\time_*\.csv] sourcetype=DGC_TIME The problem I see now is that none of my expected sourcetypes are assigned. Instead, I get csv, csv1, csv2, etc... for sourcetypes. I suspect the issue is with my regular expressions I have used in props.conf. From everything I have read, these look like they are correct, but I haven't been able to figure out what I am missing. Does any have any suggestions about my approach, and/or what might be wrong with my regular expressions? Thanks ... View more

I have an odd problem with time extraction from some CSV files. I specify the time format using the following: TIME_FORMAT = %m:%d %Y %H:%M:%S:%Q and it generally works. I am not convinced it is the correct format to use, but it seems to work(most of the time). It works fine with times like this: 07:30 2013 12:59:57:0 and it does not work with times that look like this: 01:01 2000 00:00:03:0 With these times, it seems that splunk chooses to use the file creation time(actually its a bit more complex than that - from what I can tell, it used the file creation date and the time from when the file was last modified). I know that data from the year 2000 is generally not relevant, but with the data I am looking at, there scenarios where the date may not yet be properly initialized and this is where the time will startup. I want to be able to filter these invalid dates out using the time specification, but do not want to throw away the data with invalid dates(it may hold useful diagnostic value). I suppose one answer to this might be that Splunk does not recognize times from the year 2000, but I am not sure why that would be. BTW, this data is being monitored on a windows machine with a Universal Forwarder and the indexer is on a Linux machine. Can anyone explain what might be happening here? ... View more

I know that this question has been asked quite a few times, but I have not been able to resolve this. Can I monitor multiple sourcetypes from the same path? The answer seems to be yes, but this just wont work for me. If I specify two monitors that reference the same directory, I only get data from the second monitor stanza in my inputs.conf. I suspect there is something subtle happening here I have not picked up on. Does anyone have any suggestions? Below is an example of the inputs.conf I have been using. [monitor:///home/bob/time_data.csv] sourcetype = DGC_TIME index=main [monitor:///home/bob/pulse_data.csv] sourcetype = DGC_PULSE index=main ... View more