Getting Data In

Problem to forward data from W7 to splunk(set on ubuntu VM) with Splunk universal forwarder

Federica_92
Communicator

Hi all, I have this situation:
I have installed splunk universal forwarder to forward the logs of Windows seven, to my splunk in ubuntu.
The universal forwarder works good, and use the port 8089, its config file input and outputs (set in c:\ ..system/local..) are these:

Input:

[default]
host = FROSSI-LT
[WinEventLog://Security]
disabled = 0
[WinEventLog://Application]
disabled = 0
[WinEventLog://System]
disabled = 0
[perfmon://FreeDiskSpace]
disabled = 0
[perfmon://Memory]
disabled = 0
[perfmon://LocalNetwork]
disabled = 0
[perfmon://CPUTime]
disabled = 0

Output:

[tcpout]
defaultGroup = default-autolb-group
[tcpout:default-autolb-group]
server = 10.28.4.154:9997
[tcpout-server://10.28.4.154:9997]

Where 10.28.4.154 is the IP of my ubuntu VM, and 9997 is open and listen. Enable from the splunk platform.

The same conf file in ubuntu VM are these:
input:

[default]
host = ubuntu
[tcp://:8089]
connection_host = 10.28.4.143
source = tcp:8089

Where 10.28.4.143 is the IP address of my Windows7

output:

[tcpout]
defaultGroup = default-autolb-group
[tcpout:default-autolb-group]
server = 10.28.4.143:9997
[tcpout-server://10.28.4.143.1:9997]

And, yeah, windows doesn't send data, and this is the error message:
"forwarding to indexer group default-autolb-group blocked for N seconds."

And later so much time, the message became:
"skipped indexing of internal audit event will keep dropping events until indexer congestion is remedied. Check disk space and other issues that may cause indexer to block"

0 Karma

Federica_92
Communicator

I have tried to follow all your step, but "no results found"

0 Karma

Federica_92
Communicator

I have tried to reinstall splunk and follow all your step, but "No results found"

0 Karma

jimodonald
Contributor

I just walked through the process and these are my notes from doing so.

This is from a fresh install of Ubuntu 14.04 and a Windows 2008 R2 VM.

Splunk 6.2.1 was downloaded in a DEB package and installed to the Ubuntu VM with the command 
 sudo dpkg -i splunk-6.2.1-245427-linux-2.6-amd64.deb

Splunk was then started for the first time with
sudo /opt/splunk/bin/splunk start

Read though the license agreement and enter "y" when prompted.

Start up your web browser and connect to your Splunk instance at http://ip.address:8000

Enter "admin" and "changeme" for the username and password combination. Change it to something you know when prompted.

You should now be at the Splunk home screen.

Click on the Settings menu and then click "Forwarding and Receiving" from the list of options.

On the Forwarding and Receiving page, click "Configure Receiving". This will show the current receiver settings, if any, and allow you to create a new one.

Click the "New" button and enter "9997" to the "Listen on this port" box and then Save.

Splunk is now listening for forwarders on port 9997.

Nothing else needs to be done on the indexer/search head.

Validate that there are no events in Splunk by searching for “*” over all time.
alt text

Turn your attention to your forwarder. I am assuming the Spunk Universal Forwarder is already installed.
Create or Edit the inputs.conf file in $SPLUNK_HOME\etc\system\local. It should look like this:


[default]
host = windows_hostname
    
    

[WinEventLog::Application]
    
checkpointInterval = 5
    
current_only = 0
    
disabled = 0
evt_resolve_ad_obj = 1
    
start_from = oldest

[WinEventLog::Security]
    
checkpointInterval = 5
    
current_only = 0
    
disabled = 0
    
evt_resolve_ad_obj = 1
    
start_from = oldest
    
    

[WinEventLog::System]

checkpointInterval = 5
    
current_only = 0
    
disabled = 0
    
evt_resolve_ad_obj = 1
    
start_from = oldest

Create or Edit the outputs.conf file in $SPLUNK_HOME\etc\system\local. It should look like this:


[tcpout]
    
defaultGroup = default-autolb-group


[tcpout:default-autolb-group]
    
server = 192.168.84.145:9997
    
    

[tcpout-server://192.168.84.145:9997]

Restart the Splunk Universal Forwarder.

Go back to the web interface on the Splunk indexer/search head and re-run your search. You should now see events from Windows.

0 Karma

Federica_92
Communicator

Yeah I have done it!

0 Karma

Federica_92
Communicator

But doesn't work in any case. For now I have not touch the config file in splunk, only the files in splunk forwarders

0 Karma

jimodonald
Contributor

have you tried to verify connectivity from the windows VM to the ubuntu vm?

ping ubuntu_ip_address

if that works, then try connecting to port 9997 on the ubuntu vm with:

telnet  ubuntu_ip_address 9997
0 Karma

Federica_92
Communicator

Work : ( the port and the IP are ok

0 Karma

jimodonald
Contributor

One thing I forgot to add.... the IP address in outputs.conf should be updated to the IP address of your Splunk indexer/search head.

0 Karma

Federica_92
Communicator

I have no idea If this could help anyone, but I have solve simple enabling my websplunk service, but the forwarder have problem yet.

0 Karma

jimodonald
Contributor

When you reinstalled splunk, did you remove the /opt/splunk directory first, or just re-install splunk on top of what was there?

if you re-installed on top of what was there, Splunk would have kept all the modifications you made to config files in the "local" directories.

0 Karma

Federica_92
Communicator

Ok, I have tried, I have delete the outputs.conf and edit the inputs.conf, only in my ubuntu VM, but now, when I try to open SPLUNK, it doesn't start, the message on terminal is the same of the splunk forwarder in windows:

Stopping splunkd...
Shutting down. Please wait, as this may take a few minutes.
.
Stopping splunk helpers...
Done.
Splunk> The IT Search Engine.
Checking prerequisites...
Checking mgmt port [8089]: open
Checking configuration... Done.
Checking critical directories... Done
Checking indexes...
Validated: _thefishbucket msad perfmon sos sos_summary_daily windows wineventlog winevents
Done
Checking filesystem compatibility... Done
Checking conf files for problems...
Done
All preliminary checks passed.
Starting splunk server daemon (splunkd)...

Done

And is splunk is not available at ubuntu:8000, What can I do?

0 Karma

jimodonald
Contributor

First thing I see is that you should not need an outputs.conf on the indexer. The outputs.conf tells Splunk where to send it's logs. In this case you're telling your indexer to send the logs back to the windows 7 computer. So remove the outputs.conf on the Ubuntu VM.

Second, your inputs.conf on the Ubuntu should identify the protocol and port the indexer is listening on. for the case you've described, this should be sufficient for your inputs.conf.

[splunktcp:9997]

Make those changes on the Ubuntu server and restart Splunk on it.

0 Karma

Federica_92
Communicator

Ok, I have tried, I have delete the outputs.conf and edit the inputs.conf, only in my ubuntu VM, but now, when I try to open SPLUNK, it doesn't start, the message on terminal is the same of the splunk forwarder in windows:

Stopping splunkd...
Shutting down. Please wait, as this may take a few minutes.
.
Stopping splunk helpers...
Done.
Splunk> The IT Search Engine.
Checking prerequisites...
Checking mgmt port [8089]: open
Checking configuration... Done.
Checking critical directories... Done
Checking indexes...
Validated: _thefishbucket msad perfmon sos sos_summary_daily windows wineventlog winevents
Done
Checking filesystem compatibility... Done
Checking conf files for problems...
Done
All preliminary checks passed.
Starting splunk server daemon (splunkd)...
Done

And is splunk is not available at ubuntu:8000, What can I do?

0 Karma
Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...