I plan to install the eStreamer app on my Heavy Forwarder to collect the logs from the Sourcefire management console and install it on the Search Head to use the app dashboards.
Do I also need to install the app on my indexer cluster, or can I just create the estreamer index?
Is there anything in the app that is indexer specific besides creating the index estreamer? I did not see anything myself.
I took a gamble and installed the eStreamer app in my Splunk environment. Turns out, it is fairly easy to setup and get working.
Environment == Running Splunk 6.2.4 with Defense Center 22.214.171.124 (build 55), Indexers are clustered. Two indexers with a master controller.
Install on the Indexers
Step 1. I extracted the eStreamer app into the master-apps directory on my master controller.
Step 2. I modified the indexes.conf file to set additional settings on the eStreamer index such as bucket size, frozen directory, and data retention. Note: Be sure to create a local directory, and copy the indexes.conf file there. Edit the indexes.conf file in the local directory.
Step 3. Deploy the new app to the cluster.
The eStreamer data collection process, I setup on my Heavy Forwarder
Step 4. I installed the eStreamer app via the UI
Step 5. Read the eStreamer documentation. eStreamer Documentation Most likely you will have to install the additional PERL modules. I had to install some of additional modules such as perl-NetAddr-IP.x8664.
Step 6. I created the required pkcs12 file, on the Sourcefire Defense Center and configured the eStreamer. The "create client" button is found in System/Local/Registration, then click on the eStreamer tab.
Step 7. I uploaded the pkcs12 file to the Heavy Forwarder to the eStreamer app local directory. You will need to create the local directory.
Step 8. I then completed the configuration via the UI for the eStreamer Client. I selected all the logging options. Be sure to also check, Verbose, debug-style logging check box.
Step 9. Watch the estreamerdebug.log file, it will tell you if the heavy forwarder is successfully connecting the the Defense Center. The errors are self explanatory. I had an issue with the password I set on the pkcs12 file. I had special characters in the password. I think PERL was having an issue submitting the special characters, from the configuration file. The password is stored in clear text, so be sure to use something not used anywhere else. The pkcs12 file also works without a password.
Step 10. See if data is showing up in the eStreamer index.
Last but not least, install the app on your Search Heads I do not have search head clustering
Step 11. I installed the eStreamer app via the UI
Step 12. Set the Security Settings of the app. Change Write access to Admin or your preferred role, at least uncheck Everyone. Also, choose All apps, for "Sharing for config file-only object if you want to share the field alias outside the eStreamer app, such as the Common Information Model.
Step 12. Launch the eStreamer app.
You should see your data.
Search Head Issue. Cannot see the eStreamer Client Status on the Search Head
This is because the Search Head is not running the eStreamer collection script.
This is the image you will see. The good news is, it can be fixed.
Step 1. Edit the Dashboard Panel.
Step 2. Edit the search string for the Panel
Step 3. Insert the hostname of your Heavy Forwarder in the search string.
SfeS-client-check-logs | eval state=case(status_id=-1,"Error", status_id=0,"Disabled", status_id=1,"Running", status_id=2,"Running", status_id=3,"Stopping", status_id=4,"Restarting") | table state
| eval state=case(status_id=-1,"Error", status_id=0,"Disabled", status_id=1,"Running", status_id=2,"Running", status_id=3,"Stopping", status_id=4,"Restarting") | table state
Step 4. Click Save
Step 5. Click Done
Your result will now look like this.
Common Information Model Issue
If you are running the common information model (CIM), also installing the Splunk Add-on for Cisco FireSIGHT will cause an conflict in the props.conf file for field alias. For example, eStreamer maps event field "msg" to "signature" for CIM functionality. With the Splunk Add-on for Cisco FireSIGHT app enabled, some of the eStreamer field alias mappings will not work. Hence, the CIM mode IDS Attacks/Network Intrusion Detection Data Model will not work properly. I know it is a file inheritance issue, I have not be able to solve it yet. I get to have some btool practice. For now I have the Splunk Add-on for Cisco FireSIGHT app disabled. I will have to get it working once I get Enterprise Security approved.
@sjaworski so ,there is no need to install Firesight TA anywhere if we dont have splunk ES? and is the app itself without any TA enough to get logs in this way?