About SplunkExplorer

SplunkExplorer

Hi Splunkers, I have to create some modular inputs that ingest data with REST API. Searching on doc, I found Configure Data Collection using a REST API call that explain very well basic settings working with Addon Builder. Using some APIs found on a Youtube tutorial, I developed with no problem data ingestion in a very simple scenario, which is using a GET method without authentication. On above doc, I got this message: For advanced data collection, create a modular input by writing your own Python code. Now, of course I'm not here to ask how to deploy an advanced scenario; my question is to be sure that a possible scenario I have in mind must be managed with custom Python code and cannot be managed, not totally as well, with AddonBuilder GUI. This is the possible scenario: The log sources that support API, demand a first API call. This call is a POST with username and password. The log source respond with an header that contain API Token User must then perform a GET with header enriched with above API Token. Now, in a "lazy" way, I could perform the first call on an API Client, like Postman, get the API Token and pass it to my addon Data Collection configuration, for example here: In this way, I can avoid totally Python code and use only AddonBuilder GUI. Simple, isn't it? By the way, following this method, the token is hard coded and if we need to change it, for example using a different username an password, we need to change the addon in Configure Data Collection Setting. What I want to achieve is: if a user want to install and use the addon on its own Splunk instance, with its own credentials, and he/she don't want provide the API Token but username and password, I have to implement this setting with Python code; I have no way to tell to Addon Builder "Ehy, take credentials and then the token is the one from response" ONLY with GUI. Right?

SplunkExplorer

It works! Thanks a lot.

SplunkExplorer

Hi Splunkers, I have a doubt about a custom app customization. For a customer, we created with Splunk Addon Builder a simple app to use as "container": every customization we perform, such as Correlation rules, reports and so on, is assigned to this app. So, in its first release, the app has no particular panel, features and so on; let's say that just "exist". To be clearer: if I login and open the app, what I see is this: and that's totally fine, due we did not perform any kind of customizations. So now, the question is: if I want to include the search function inside this app, how I can achieve this? I mean, we want avoid, when when we need to perform a search, to go on Search and Reporting app; we would be able to perform searches inside our app. For now, we don't need panel with specific charts, based on particular query: we want simple to be able to use (if it is possible of course) the Search and Reporting app/its functionality inside our app.

SplunkExplorer

Thanks all you guys, you gave me a lot of hints and you have been very helpfull

SplunkExplorer

Thanks a lot, it works! Just another last question: what about perform the change to parsing/addon? I mean: command you shared with me works if I put it on a search when I'm logged on console. Suppose customer ask us "I want data already in JSON when I perform search, withou put command | tojson". Is it possible to configure some parameter in TA_windows addon to achieve this? Or I can only get the 2 format XML and Legacy?

SplunkExplorer

Hi @PickleRick , forgive me, I fear I explained myself bad. Windows logs are coming directly from Domain Controllers. They are ingested using UF and they transitate through HF, so the final flow is: DCs with UF installed -> HF -> Splunk Cloud environment In addiction to this, the TA_windows is installed on both HF an Splunk Cloud. So, we don't want ingest data from third party forwarder; we want to know if, with this environment and the above addon installed, we are able to see logs on JSON format, when we perform searches on SH, or we can see only Legacy and XML one because, with this environment and this addon, no other format are supported.

SplunkExplorer

Hi Splunkers, I have a request by my customer. We have, like in many prod environments, Windows logs. We know that we can see events on Splunk Console, with Splunk Add-on for Microsoft Windows , in 2 way: Legacy format (like the original ones on AD) or XML. Is it possible to see them on JSON format? If yes, we can achieve this directly with above addon or we need other tools?

SplunkExplorer

Hi @tej57 , thanks a lot, you are right, searched parameter is there. I noted a strange behavior and I don't now if you can help me on this: I noted that, before change the parameter, if I search on out Splunk, events are most but not all in xml format: That seems strange because, analyzing the inputs.conf on addon, all stanza has the parameter equals to true; so, how can be possible this? My suspect is that, in another inpust.conf or .conf file, that parameter is set to true. How can I check this? May be usying the btool utility?

SplunkExplorer

Hi Splunkers, in our environment we are collecting Microsoft Windows logs that, currently, come in xml format. Customer demand us to switch off xml: on Splunk Console, he want to see logs in legacy/traditional format and not xml one. I don't remember how this can be achieved; am I wrong or I have to change a parameter in addon configuration? By the way, those are essential data for our scenario: Collection mode: UF installed on each Windows data sources. Logs are sent to an HF and then to a Splunk SaaS, so final flow is: log sources (with UF) -> HF -> Splunk SaaS (both Core and ES). UF management: all UF are managed with a Deployment Server Addon used: the "classic" one, https://splunkbase.splunk.com/app/742 Addon installation: on both SaaS env and HF

SplunkExplorer

Hi Splunkers, in the end I worked with Support and it figured out the reason of not working regex: when applied to a monitor input on a UF, like that case, the blacklist parameter is applied to file name and/or path; our purpose is to filter out based on file payload and, to achieve this, we must work on HF, changing inputs.conf or both props and transforms.conf.

SplunkExplorer · ‎11-07-2023

Hi Splunkers, in our Splunk Cloud environment we had 2 need: Reassign knowledge object owner Reassign Knowledge object app The first point management is well known and we already applied it, assigning all KOs created by us to a service account. I don't remember if the second one is possible: can I reassign a KO app? For example, if we assigned an alert to search app, is it possible to change it with another one? And if yes, how?

SplunkExplorer · ‎10-06-2023

I fear I explained myself in a bad way Giuseppe, sorry. Our purpose is to filter out only the case when domain you can find in parenthesis have a "proper" form. For example: microsoft.com azure.net office.us Those domain for us are admitted ones, so we don't need to see them on SIEM and we want avoid that UF send logs with them on SPlunk Cloud. On the other side, if the domain is "strange", like: microsoft.123.com azure-pirate.com office.tryhackme.us we want to be alerted and so, in this scenario, logs must be sent to Splunk. Now, based on this, if you see Regex I shared with you, the normal behavior may be: Case 1: regex matched -> Logs NOT send to Splunk Case 2: regex NOT matched -> Logs MUST be sent on Splunk. So, what the problem? The logs for case 1 has been sent to Splunk, even if it match the regex and so it should be discarded. In other words: the log of Regex Matched contain "microsoft.com", match the regex, should be discarded but it has been sent anyway to Splunk.

SplunkExplorer · ‎10-06-2023

Old format, no XML

SplunkExplorer · ‎10-06-2023

Hi Giuseppe, below the link to regex101 with a used regex and a log that match it: Matching regex Here same things but with a little change to log that made it not matching the regex, like expected: Not matching regex Another idea is my use of capturing groups; should I use them in another way?

SplunkExplorer · ‎10-06-2023

Hi Splunkers, I have a problem with a blacklist filter. On customer's UF, we filtered out some events changing the inputs.conf file. The ones based on comma separated list, like Windows EventID, are working fine with no problem, while the one based on regex not. Of course, as first thing, I checked regex syntax and I can confirm it works fine; testing it on regex101, it match perfectly what I want. Tests have been with different source logs, to be sure of a full proper working. This is how we placed regex on UF: [<stanza name>] ...other parameter... blacklist = \]\sA\s+(.*)(microsoft|office|azure|o365|onenote|outlook|windowsupdate)(\(\d+\))(com|net|us)(\(\d+\))\s This filter must be applied to logs coming by Windows DNS; its purpose is to avoid ingestion of legit domain, in all their combination, but only if they have a "normal" form. In regex you can see I put a filter about (<number>), because in raw log we have domains in format main_domain(<number>)root_domain, like microsoft(3)net. For example, microsoft(2)com and microsoft(3)net match the regex and should be filtered out, while microsoft(9)123(5)com not and should be sent to Splunk. My assumption is that I missed out some delimiter after the equals symbol; I mean, should I put regex code between any kind of symbols? Something like regex = '<regex code'> Or regex = "<regex code>" etcetera.

Posts	75
Solutions	4
Karma Given	28
Karma Received	6
Member Since	‎09-22-2022

Online Status	Offline
Date Last Visited	Tuesday

Splunk REST API advanced data collection

Add Search bar to custom app built with Add On bui...

Convert ingested WinEventLog in JSON format

Windows logs: switch from xml format to legacy one

Reassign Knowledge Object app

UF inputs.conf blacklist rege based not working

UF checks on file open when "writing"

How to end File monitored data input from HF to Sp...

Splunk monitor data input different behavior after...

How to send splunk alerts to Jira?

Splunk REST API advanced data collection

Re: Add Search bar to custom app built with Add On...

Add Search bar to custom app built with Add On bui...

Re: Convert ingested WinEventLog in JSON format

Re: Convert ingested WinEventLog in JSON format

Re: Convert ingested WinEventLog in JSON format

Convert ingested WinEventLog in JSON format

Re: Windows logs: switch from xml format to legacy...

Windows logs: switch from xml format to legacy one

Re: UF inputs.conf blacklist rege based not workin...

Reassign Knowledge Object app

Re: UF inputs.conf blacklist rege based not workin...

Re: UF inputs.conf blacklist rege based not workin...

Re: UF inputs.conf blacklist rege based not workin...

UF inputs.conf blacklist rege based not working