Indexing Huawei Firewall device logs into Splunk

[eidil]

Hi,

I am trying to ingest huawei USG6650 device logs but it seems that no app is available in splunk base for this purpose. Is there any other way/guide for this?

[richgalloway]

There are several ways to get data into Splunk.

1) Install a Universal Forwarder on the device to send logs to Splunk

2) Have the device send syslog events to Splunk via rsyslog, syslog-ng, or Splunk Connect for Syslog

3) Write a script that uses the device's API to extract data and index it in Splunk

4) Use Splunk DB Connect to extract data from the device's SQL database

5) Have the device send events directly to Splunk using HTTP Event Collector (HEC)

Which one you use will depend on the device and it's capabilities.

[eidil]

Sorry for the confusion, but my intention is to know how the huawei device data can be indexed in splunk and populated with important fields/formats that can be used for such as ESS app.

seems that for cisco appliances, you can use TA-cisco-app for the logs to be populated with important fields. Any alternatives for huawei devices?

[richgalloway]

Yes, and I listed five such alternatives. In each, however, you must extract fields yourself.

[inventsekar]

Hi @eidil .. some devices, like cisco firewall, will have apps/add-ons, technical addons, etc,.. they are generally created by Splunk or Device owners themselves/or some Splunk Consultants, Engineers, Developers. 

in the same way, some devices,..generally new or not famous devices, i know huawei devices are famous, but it may happen at times that, these devices may not have addons on SplunkBase. 

so, the alternative approaches are - As @richgalloway suggested clearly. 

As per understanding, firewall devices are good candidate for Syslog - Splunk integration. 

[nedra]

Regarding the same topic, please, for Huawei equipment, is there no automatic method or recommended method for parsing?

[PickleRick]

Existence of pre-made add-ons depends highly on the solution importance/popularity/market share etc.

Vendors will invest their time and resources into supporting a solution only if it's "significant" enough. People will make community-driven add-ons only if the solution is popular enough (and they are not bound by some contractual clauses so that even if they create something in-house they cannot share it).

This is true for every product on the market, regardless of whether we're talking in Splunk's context or any other product.

So apparently this is one of the solutions that didn't hit the threshold of popularity/importance so no ready-made solutions exist.

In Splunk's case however you're not limited to out-of-the-box integrations. You can do stuff manually even if ready-made solutions don't exist.

You have two problems to tackle.

First is how to get data from your source (in your case - the firewall) to Splunk. With network equipment it's usually the syslog method. Ingesting syslog is a typical task and there is plethora of information all around the internet about it.

Second is parsing - you have to parse specific fields from the events. This can be more time-consuming to do properly but can be done in several ways - manually using props/transforms, via Splunk's webgui, ussing Add-on Builder...

So it's not that just because there is no ready-made add-on, the data can't be used.