I have a network share folder with a huge number of directories and files (.csv). Files are constantly being added and periodically getting removed for archiving. When creating a Directory data source for this share I found that splunk is opening a lot of file handles as it tries to watch the files and monitor for changes. When restarting my instance it also takes a very long time for this monitoring to begin working; I was told this is because it needs to "catch up" with any missed changes by scanning the entire share. So I tried using splunk's HEC to send my .csv files. If I try to send each row of the csv as an individual event (source type=_json) then the overhead of repeating the csv headers for each row quickly builds up (these csvs are often very large and have >40 headers (the headers are also not very static)) and it unnecessarily burns through the license limit. It does work though. If I try to send the raw string content of the csv file as a single event (source type = csv) it doesn't interpret it correctly (it doesn't detect the fields). I'm not even certain this is supposed to work in the first place. So both the Directory data source and HEC seem to be inefficient for my scenario. Are there any other options I can try (out of the box preferably or an official app)? Or perhaps tweaks to above the methods (preferably not undocumented settings)? ... View more

I am trying to create a role that has access to only a single index and can only view the 'search' app. The way I created the role was by copying all the capabilities and other settings from the 'user' role to my new role. The only differences are that the 'Indexes searched by default' and 'Indexes' list are limited to only the one index I want them to see. I then went to the 'Permissions' page for the Search app and gave the new role Read+Write permissions. After creating a dummy user and placing it in the role, I logged in and found that indeed it only had access to the search app and could not see others. However when I attempt to execute a search, no results are returned. The search query I used was: 'index=my_custom_index'. The same query works if I run it as myself (an admin). My splunk set up is: 3 search heads, 3 forwarders, 4 indexers, and a license server. I made all of the above changes on the captain search head's UI. Are there any steps that I am missing? And are there any other troubleshooting techniques I can use? I've tried looking at the search job logs but there are no clear indications of what permissions were missing or what caused 0 results to be returned. ... View more