How can I ingest firewall ,waf ,ssandbox ,email gateway, endpoints logs to Splunk ES datamodels?
I am trying to work on Splunk ES dashboards with the below details:
These are the sources to look for:
• network traffic : (firewall ip address) (proxy: ip address) (imsva (mail gatewa) ip address - shqimsva), ip address
• malware: ip address (apex central - index is trendmicro) proxy and mail gateway , ip address
• dlp: ip address (apex central - index trendmicro), mail gateway [DLP_Credit Card info] -
• waf = ip address
• web center (waf, iwsva proxy ip address)
Dashboards are not loading details about the above logs.
First you need to make sure your data is CIM (Common Information Model) compliant. This is to normalize the data so you are using the same field values for the datamodel. You will want to look at each data model for the Common names and then check your data that corresponds to that datamodel. https://docs.splunk.com/Documentation/CIM/4.13.0/User/Overview
Example for Network Datamodel: https://docs.splunk.com/Documentation/CIM/4.13.0/User/NetworkTraffic
The data also needs to be tagged properly. Network traffic has the tag of network and communication. This is typically done through the app but you will need to check this as well.
This is is where I would start and move through each datamodel and data source you are having issues with.
how to make sureor check that your data is CIM (Common Information Model) compliant
Search the data in question to make sure fields, for the datamodel they need to be a part, exist. Also check the for the field tag. The data needs to be tagged appropriately for that data model as well (tag=network and communicate). So your firewall data should follow the Network traffic data model. If the firewall has an IPS/IDS and that data is sent to Splunk, it will need to follow the datamodel Intrusion Detection field names and tags=ids and attack.
Thanks for the info but i tried malware dashbnoard with CIM malware index and tag =malware tag=attack....however it doesnot shows the malware detected by trendmicro deep discovery analyzer.
i have checked app as smtp , signature ,policy but no way i am tired of this datamodels even though mu dashboard like
network resolution DNS
dashboards are showing some data but i need to have real time datas from deep discovery analyszer , trendmicro apex central, firewalls , fortinet , trendmicro waf....
not sure what to do ....also because of ES now my indexer has also started crashing...
If you are having issues with the stability of your environment, I would open a Support case.
can you give an example showing how do you map data in datamodels using your network devices as source.
for ex:source is trendmicro, data model is malware how can you configure the datamodel.
When you search your trend micro data, does this data have a tag field? You will see this on the left hand side when searching your data as Selected fields or Interesting fields. If you are having issues with your Splunk server, I would recommend calling support and asking for assistance in making changes to your datamodels as this can impact the performance of your Splunk environment.