Getting Data In

Struggle to get Syslog data parsed out for network device logs.

antoniolamonica
Explorer

Our network device data sends data to a Syslog server and then up to our splunk instance. 

I have a few TAs that I’ve requested our SysAdmin team install on my behalf so the logs can be parsed out. 

However, the TAs aren’t parsing out the data, and furthermore, the network device logs come into the source type of “syslog” rather than the sourcetype in the respective TAs. 

Where do I need to look or have the SysAdmins look at?

(I’m just a power user). 

0 Karma

PickleRick
SplunkTrust
SplunkTrust

It depends on your overall syslog-ingesting process. As you're saying that "device sends data to a Syslog server and then up to our splunk instance" I suppose there is a "middle-man" in form of some syslog receiver either pushing the data to HEC input or writing to files from which the data is picked up. In this case it depends on that "middle-man" configuration.

If however it's just a case of a bit imprecise wording and all your devices send directly to your Splunk component, you have to make sure that you have proper inputs configuration on that box (and proper sourcetype configs as well). As a rule of thumb you can't have several different sourcetypes on a single tcp or udp port input with Splunk or Universal Forwarder alone.

0 Karma

antoniolamonica
Explorer

Sorry for the vagueness; the imprecise wording is intentional due to the nature of the environment I work.

The network devices' logs get sent to a syslog server. 
The syslog server writes all the logs to file in a specific path.

On our Server Class server, the Data Input settings is configured to read all the files from that path (its a unique enough path) and sends it to our "network_devices" index.  

So the data is being sent to the correct index, but a good portion of the logs are sent to the sourcetype=syslog, rather than the TA's sourcetype.

That is where I am stuck. 

0 Karma

PickleRick
SplunkTrust
SplunkTrust

OK. So if you split your events received by syslog into separate files based on the source device, you should configure your monitor inputs to pick different kinds of files with specific sourcetypes so you don't ingest the whole big directory with all your "network" logs but instead fine tune it with subsets of the files pertaining to specific devices.

If you're saving all syslog-received events to one big file - that's way harder because you can only associate one sourcetype for a given monitor input. You might try to later dynamically overwrite it during ingestion process using props and transforms but this will be way way harder than doing the splittin on the syslog-receiver level.

0 Karma

antoniolamonica
Explorer

I see your point. 
It’s not necessarily 1 big file. 
The files are do share a base name, but is then automatically appended with the Unix time in the file name. 

But to your point, I could reach out to the SysAdmins if they could have the file names be associated by network device family, and then append with the Unix time.  Or create a sub directory in that path named for each network device family. 

 A little more granular, but I’m not sure what they can do on their end. 

0 Karma
Get Updates on the Splunk Community!

Detecting Brute Force Account Takeover Fraud with Splunk

This article is the second in a three-part series exploring advanced fraud detection techniques using Splunk. ...

Buttercup Games: Further Dashboarding Techniques (Part 9)

This series of blogs assumes you have already completed the Splunk Enterprise Search Tutorial as it uses the ...

Buttercup Games: Further Dashboarding Techniques (Part 8)

This series of blogs assumes you have already completed the Splunk Enterprise Search Tutorial as it uses the ...