topic Re: Is it possible to monitor the application logs with out agent using Splunk? in Monitoring Splunk

Is it possible to monitor the application logs with out agent using Splunk?

Hemnaath — Mon, 25 Nov 2019 05:32:18 GMT

Hi All,

Currently we have BMC-True site application monitoring the application logs using an agent, but we wanted to move forward with an agent-less monitoring in future, not only just monitoring the device but we wanted to use the data to improve our performance.

Please do let me know how we can use Splunk to do this better than the current application.

Thanks in Advance.

Re: Is it possible to monitor the application logs with out agent using Splunk?

gcusello — Mon, 25 Nov 2019 07:56:10 GMT

Hi @Hemnaath,
Yes with Splunk you can take logs also without agents using other ways as syslogs or WMIs or HEC, but I usually discourage my customers from doing so because the presence of the Universal Forwarder (the Splunk agent) provides many operational, security and guarantee advantages of not losing anything.

In short, the most relevant advantages are (but not the only ones):

UF caches when there is no connection with the center so you don't lose logs (syslogs are in real time and in case of problems you lose logs);
The UF optimizes the traffic by compressing the logs (syslog no);
UF can do log encryption (syslog no);
The WMI is invasive and a rich administrative grant (which is not recommended for security reasons);
with the UF you can decide what to take and therefore limit the network effort while the syslog sends everything and must eventually be filtered on arrival;
if you want to add other logs, you can manage this request centrally using the Deployment Server, with syslog you have to manage this request server by server.

In other words, I usually use UFs, I never use WMI and I use syslogs only when I cannot do otherwise (appliances).

Ciao.
Giuseppe

Re: Is it possible to monitor the application logs with out agent using Splunk?

Hemnaath — Tue, 26 Nov 2019 07:41:13 GMT

Hey thanks for the valuable information, but how do you monitor log files using the HEC collector, We need to create an POC for the same to understand it better. Is there How to do ? document or steps for capturing log file using the HEC in splunk.

Re: Is it possible to monitor the application logs with out agent using Splunk?

gcusello — Tue, 26 Nov 2019 09:03:21 GMT

Hi @Hemnaath,
HEC is only for custom application where it's possible to push http/https messages.
You can find docs and videos at:
https://dev.splunk.com/enterprise/docs/dataapps/httpeventcollector/
https://docs.splunk.com/Documentation/Splunk/8.0.0/Data/HECExamples
https://www.youtube.com/watch?v=9awwyjORWO8
https://www.youtube.com/watch?v=qROXrFGqWAU

Ciao.
Giuseppe

Re: Is it possible to monitor the application logs with out agent using Splunk?

Hemnaath — Thu, 28 Nov 2019 06:13:40 GMT

Hi Gcusello, thanks for your support on this, I had gone through the all the above splunk videos. But I need to implement this practically to understand this concept better. Is there any example which you can share, so that I can create a POC.

Thanks in Advance.

Re: Is it possible to monitor the application logs with out agent using Splunk?

gcusello — Wed, 30 Sep 2020 03:09:49 GMT

Hi @Hemnaath,
what do you want to monitor: Windows, an application an appliance ?

To easily start you could install Splunk Enterprise on a server (for a PoC you can use Windows or Lunix, but for production I hint to use only Linux) and then Universal Forwarder on a Windows Server.

Then you have to enable Splunk to receive logs on 9997 port and UF to send logs to Splunk.
You can test it searching on Splunk index=_internal host=uf_hostname.
For more infos see at https://docs.splunk.com/Documentation/Splunk/8.0.0/Data/Getstartedwithgettingdatain

Then you can take the Splunk_TA_Windows and then untar and copy it both on UF Splunk at $SPLUNK_HOME\etc\apps.
then on UF you should edit inputs.conf, copy inputs.conf from default folder to local folder and edit it changig disabled=1 to disabled=0 to all the thing you want to tale (e.g. security events).
at the end you have to restart both the splunks.

In this way you'll have the windows logs and you can see them with a simple search like index=wineventlog.
You can use these logs for your use cases:
e.g. if you want an example of brute force attack with a number of logfail greathen than 10 you could create a search like this

index=wineventlog EventCode=4625
| stats count BY host
| where count>10

Another interesting thing to do in a PoC is to install an app like Splunk App for Windows Infrastructure ( https://splunkbase.splunk.com/app/1680/ ), follow the install instructions!
Another interesting use case is to install the UF on a Domain Controller following the instructions of the above app and monitoring the Domain.

Ciao.
Giuseppe

Re: Is it possible to monitor the application logs with out agent using Splunk?

Hemnaath — Fri, 29 Nov 2019 16:36:41 GMT

thanks for your valuable time and input on this.

Re: Is it possible to monitor the application logs with out agent using Splunk?

woodcock — Fri, 29 Nov 2019 19:11:39 GMT

You can use just about anything including:

WMI
Http Post (e.g. HEC)
Snare
Kafka
syslog (UDP/TCP)
cribl
NiFi

I also know that @duanewaddle @dwaddle has been thinking about building something in this area and probably has strong opinions here.