How can we read local data on HF and send them to indexers?

[kaboom1]

Hello,

Here is the deal, I am following this link to ingest cisco umbrella logs into splunk: https://support.umbrella.com/hc/en-us/articles/360001388406-Configuring-Splunk-with-a-Cisco-managed-... (I know, you will say: why not use one of the existing apps in SplunkBase? I will say: I really don't know which one to use, they seem doing the same thing, plus I just want the raw logs, so getting the logs directly from the bucket is good enough 🙂 )
I downloaded the logs and stored them in the HF under /opt/ciscologs/  and i configured a simple inputs.conf file to read those files with the idea that these logs will be sent to the indexers (because I already have the output file with the config to send data from HF to indexers) 

But, I dont see any logs being indexed, I dont see any events on the search head 😞 

here is the inputs.conf in the HF:

[monitor:///opt/ciscologs/dnslogs/*]
index = index_name
sourcetype = csv
#whitelist = 2023-*/*
disabled = 0
crcSalt = <SOURCE>
#_TCP_ROUTING = default-autolb-group

(the commented lines are settings that i have tried but still no luck.)

Any suggestions here? I am out of ideas 

Thanks.

[richgalloway]

Did you restart the HF after creating the inputs.conf file?  The changes don't take effect until restart.

[kaboom1]

I didnt restart the HF because I am managing the conf files from a deployment server.

[richgalloway · Accepted solution]

The HF still must restart to force it to re-read the config files.  The DS has an option that tells the client to restart when it gets an update to the app.  Be sure that option is selected.

[kaboom1]

Thank you @richgalloway , indeed by simply selecting the option Restart Splunkd from the deployment server GUI (in Settings > Forwarder management > select your app then click Edit abd select: After installation: Restart Splunkd)  logs start coming in 🙂 

[isoutamo]

One way to check when UF has restarted

index=_internal host=<your UF name known by splunk> sourcetype=splunkd source=*splunkd.log component IN (ModularInputs ExecProcessor TailingProcessor)
| rex "New scheduled exec process: (?<mod_input_script>.*)" 
| rex "Parsing configuration stanza: (?<input_file>.*)\." 
| eval input = coalesce(input_file, mod_input_script) 
| where isnotnull(input) 
| eval input = if (input == mod_input_script, "modinput:" . input, input) 
| bin span=1d _time as time 
| bin span=1h _time as bTime
| stats min(_time) as firstBoot max(_time) as lastBoot values(bTime) as BootTimes values(input) as input by time host 
| table firstBoot lastBoot BootTimes host input
| convert timeformat="%F %H:%M" ctime(BootTimes) 
| sort + firstBoot -lastBoot
| convert timeformat="%F %T" ctime(firstBoot) ctime(lastBoot)

[isoutamo]

Hi

can you give a sample of those filenames and sampled content of those files?

Have Splunk HF's splunk server process access right to those files? This instructions shows that sync is done as root and if your splunk is running as splunk-user (never run it as root), then maybe splunk-user haven't access to those files.

r. Ismo

[kaboom1]

hello @isoutamo ,
so the downloaded files have this structure: /opt/ciscologs/dnslogs/date/date-hour-text-csv.gz

for example: /opt/ciscologs/dnslogs/2023-06-01/2023-06-01-12-20-732d.csv.gz

I didnt run the script as root, I only use splunk user, so the files are owned by splunk user.