About hettervik

hettervik · ‎09-02-2022

Seems like this problem has been addressed in newer versions of the supported Splunk Windows TA. https://docs.splunk.com/Documentation/AddOns/released/Windows/Configuration#Configure_Windows_Update_Logs_in_inputs.conf

hettervik · ‎08-26-2022

In general, as notable events are stored in a separate index (called "notable"), the permission for the indexes for the original events doesn't apply anymore. However, I guess you could add the index field from the original events to the notable event, call it e.g. "original_index", and then create a search filters for different user groups, so that for example users with access to index "foobar" can only see notable events with "original_index=foobar". I'm not sure how this would work for incidents in Splunk ES though, as I understand that viewing incidents in the "Incident Review" in Splunk ES is not the same as looking up events from the notable index per se, and that search filters might not apply here.

hettervik · ‎08-26-2022

If you want index to be included as a field in the incident itself, you could add "index" as an incident review event attribute. Configure > Incident Management > Incident Review Settings > Incident Review - Event Attributes For this to work you would also have to make sure that the index field is included in the output of your correlation searches as well. This would require edits to a lot of the out-of-the-box correlation searches that use the tstats command on data models.

hettervik · ‎08-26-2022

Note that "session_id" is not an eval field in the Network Traffic data model, meaning that it could be null for some entries. If you add session_id to the by clause, these entries would be dismissed. If this is the case, you can either use the "fillnull_value" argument on the tstats command, or instead of adding session_id after the by clause, add it as a values function. | tstats summariesonly=t count values(All_Traffic.session_id) as session_id min(_time) as firstTime max(_time) as lastTime from datamodel=Network_Traffic by All_Traffic.src All_Traffic.dest All_Traffic.dest_port

hettervik · ‎08-26-2022

You haven't activated the alert throttling in the correlation search by chance? That could explain it. Alternatively, if the alert run frequently over a small timespan, e.g. run each 5 minutes and searches in the last 5 minutes, and there's also a delay in the logs to Splunk, the alert could "miss" the logs. The solution to this would be to add a delay to the search, so that it e.g. searches from -10m to -5m instead of -5m to now.

hettervik · ‎08-25-2022

Thanks. Though we have a distributed environment, so if the attacker gets access to CLI on the search head, they still cannot delete the data on the indexers. We have extra layers of protection for the indexers. Therefore it would still help to be able to disable the delete command, if I'm not missing something.

hettervik · ‎08-25-2022

In Splunk there exist a delete command. Any admin in Splunk can give themself the capability to use this command. In theory, if a single admin user in our Splunk environment is compromised, the attacker can delete all data from the Splunk indexers. I know that the data is not actually deleted from disk when using the delete command, but still it is for all practical purposes deleted. Is there any way to securely disable the delete command/capability in Splunk, so that not even administrators can get access to it? Preferably we want to disable the command on the indexer layer, so that even if the OS on the server hosting the search head is compromised the command cannot be used. Alternatively, if the command can be disabled on the search head in a way that it cannot be re-enabled through the web interface, that is better than nothing.

hettervik · ‎08-16-2022

Don't need an answer to this question, just wantet to share my solution.

hettervik · ‎08-16-2022

We've starter lookin into Risk-Based Alerting (RBA) in Splunk ES, and noticed that the logic for the risk notables is in fact case sensitive for risk objects (users and systems, mostly). This is a bit counterintuitive, as the Asset & Indentity (A&I) settings clearly says that it is are not case sensitive, but we figured out that RBA doesn't use A&I at all, and instead just used the fieldvalue for the user/system directly, without having any logic to merge users/systems under different aliases. I've made a small change to the RBA alert "Risk Threshold Exceeded For Object Over 24 Hour Period" to at least make it case insensitve, in case anyone else need a fix for this problem as well. Just change the two first lines for the search from this: | tstats `summariesonly` sum(All_Risk.calculated_risk_score) as risk_score, count(All_Risk.calculated_risk_score) as risk_event_count,values(All_Risk.annotations.mitre_attack.mitre_tactic_id) as annotations.mitre_attack.mitre_tactic_id, dc(All_Risk.annotations.mitre_attack.mitre_tactic_id) as mitre_tactic_id_count, values(All_Risk.annotations.mitre_attack.mitre_technique_id) as annotations.mitre_attack.mitre_technique_id, dc(All_Risk.annotations.mitre_attack.mitre_technique_id) as mitre_technique_id_count, values(All_Risk.tag) as tag, values(source) as source, dc(source) as source_count from datamodel=Risk.All_Risk by All_Risk.risk_object,All_Risk.risk_object_type | `drop_dm_object_name("All_Risk")` To this: | tstats `summariesonly` sum(All_Risk.calculated_risk_score) as risk_score, count(All_Risk.calculated_risk_score) as risk_event_count, values(All_Risk.annotations.mitre_attack.mitre_tactic_id) as annotations.mitre_attack.mitre_tactic_id, values(All_Risk.annotations.mitre_attack.mitre_technique_id) as annotations.mitre_attack.mitre_technique_id, values(All_Risk.tag) as tag, values(source) as source from datamodel=Risk.All_Risk by All_Risk.risk_object,All_Risk.risk_object_type | `drop_dm_object_name("All_Risk")` | eval risk_object=lower(risk_object) | stats sum(risk_score) as risk_score, sum(risk_event_count) as risk_event_count, values(annotations.mitre_attack.mitre_tactic_id) as annotations.mitre_attack.mitre_tactic_id, dc(annotations.mitre_attack.mitre_tactic_id) as mitre_tactic_id_count, values(annotations.mitre_attack.mitre_technique_id) as annotations.mitre_attack.mitre_technique_id, dc(annotations.mitre_attack.mitre_technique_id) as mitre_technique_id_count, values(tag) as tag, values(source) as source, dc(source) as source_count by risk_object, risk_object_type

hettervik · ‎07-01-2022

In your case the error message seems to be linked to a specific script, in a specific stanza, in a specific app. I would look into that particular input script, "duo_input.py". Perhaps it's using unsupported Python 2 (as opposed to Python 3)?

hettervik · ‎06-28-2022

New working link that always point to latest version 🙂 https://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/ScriptSetup

hettervik · ‎05-20-2022

On the page "Configure data collection using a REST API call" there is a section about adding setup parameters. However, on the shell input page "Configure data collection using a shell command" there is no such section. There is a section about adding input parameters, but it's not the same. The reason I'm asking is because I'm trying to add setup parameterts to a shell input, and I just get error messages in the final validation page, no matter what I do. Should it be the same syntax as for REST-inputs, or is it different for shell inputs? See attached screenshot of what I'm trying to do. I've already tried the following versions of the parameter syntax, but I get the same error messages for all of them, and yes, I've added values to the parameters (in the Add-on Setup Parameters tab). ${__settings__.additional_parameters.my_parameter} ${additional_parameters.my_parameter} ${my_parameter} Also, I get it to work if I switch to input parameters instead, but in this case I want to use setup parameters, as I'm planning to re-use the parameters in several inputs.

hettervik · ‎04-21-2022

Thanks for sharing this, works great! Could also be added in the readme that it's possible to change the destination for the lookups on the target server by changing the following lines in myvariables.py. From: SplunkLookupURL = "/services/properties/lookups" SplunkSearchURL = "/services/search/jobs" To: SplunkLookupURL = "/servicesNS/-/<yourapp>/properties/lookups" SplunkSearchURL = "/servicesNS/-/<yourapp>/search/jobs"

hettervik · ‎04-21-2022

Sorry for not updating this thread earlier. It's a while ago since I worked with this, but if I remember correctly the problem was not related to performance at all, even though the error messages might indicate so. Think this worked out after ensuring that the user account and password was correct and that there were no firewalls blocking the request from Splunk to Proofpoint. My understanding is that the Proofpoint TAP SIEM Add-on runs a test on the data input settings before saving them, which includes actually connecting to Proofpoint with the credentials, so you cannot pre-save the data input before everything else is in order as well.

hettervik · ‎03-25-2022

This is great, but is there any way of finding the "search_id" of a scheduled search? I've tried using the search_id that is listed in the URL when opening the search in the GUI and the search_is that is listed on the enpoint https://<host>:<mPort>/services/search/jobs (which I found to be not the same for some reason), but I always get the result "Unknown endpoint". Anyone know how to find the correct ID for a scheduled search?

hettervik · ‎03-15-2022

In Splunk ES we have correlation searches creating notable events. The timestamp of the notable event, and thus the timestamp of the incident in "Incident Review", is the time of when the correlation search ran. Is there any way to change this timestamp to a custom timestamp, i.e. the time of the actual log event in Splunk that triggered the notable event? I know one solution is to make the correlation search run really often, like every minute, which would make the timestamps quite precise, but not perfect, and also this would not be optimal with regards to performance. Also, I guess we could change the default time parsing of notable events in Splunk ES and add my own time field, e.g. "my_time_field", and use this field for time parsing instead, but then all out-of-the-box correlation searches in Splunk ES would stop working properly and it is in general not a good solution. We've made a temporary solution to this by adding a new "Incident Review Event Attribute" field called "Alert Time", which adds a new field to the incidents with the "real" timestamp, but it's not optimal, as the time of the incident itself is still the same. Is there any other way?

hettervik · ‎03-10-2022

We are trying to test out the new "cascading" option for the knowledge bundle replication policy. We have sat the replication policy on the search head to "cascading" in distsearch.conf, and have sat a new pass4SymmKey on the indexers for the cascading replication in server.conf, as described in the docs. https://docs.splunk.com/Documentation/Splunk/8.1.6/DistSearch/Cascadingknowledgebundlereplication However, in the Monitoring Console, on the page "Search > Knowledge Bundle Replication > Cascading Replication", the indexers still says that the replication policy is "classic", not "cascading". The search head correctly identifies as "cascading", but not the indexers. Dashboard: https://<monitoring-console-server>/en-GB/app/splunk_monitoring_console/cascading_replication How to get the Monitoring Console to correctly identify the knowledge bundle replication policy on the indexers as "cascading"? Is the check only looking for the setting "replicationPolicy = cascading"? If so, the indexers will always "fail" the check, as this setting is not applied on the indexers, as far as I understand.

hettervik · ‎02-17-2022

I've found myself coming back to this problem, and still I cannot understand how to properly troubleshoot this health alarm from Splunk ES. I think one problem with using the lookup "asset_lookup_by_str" to find mv-fields that exceeds the limits is that the mv-fields are already truncated, so it's impossible to see which fields that was actually over the limit, and which fields was on the limit but not over it. Also, out-of-the-box, some mv-fields has a limit of 25 and some has a limit of 6, so there are two different limits. I tried making a new search using the "entitymerge" command, but this also truncates the mv-fields, so I've gone back to looking at the "asset_lookup_by_str", and looking for fields that are on the limit, indicating that before the merge they could possibly have been over the limit. This is my new version of the search. For future reference, if someone needs it. I've also removed the asset fields, as I'm unsure if it actually has a limit or not, and is a cause for the alarm. | inputlookup asset_lookup_by_str | eval es_lookup_mv_limit_small=6 | eval es_lookup_mv_limit_big=25 | foreach ip mac nt_host dns [eval count_<<FIELD>>=coalesce(mvcount(<<FIELD>>), 0), es_lookup_is_problem_small=mvappend(es_lookup_is_problem_small, if(count_<<FIELD>> >= es_lookup_mv_limit_small, "yes - field <<FIELD>> possibly has over ". es_lookup_mv_limit_small . " entries", null()))] | foreach bunit category city country lat long owner pci_domain [eval count_<<FIELD>>=coalesce(mvcount(<<FIELD>>), 0), es_lookup_is_problem_big=mvappend(es_lookup_is_problem_big, if(count_<<FIELD>> >= es_lookup_mv_limit_big, "yes - field <<FIELD>> possibly has over ". es_lookup_mv_limit_big . " entries", null()))] | where isnotnull(es_lookup_is_problem_small) OR isnotnull(es_lookup_is_problem_big) | eval es_lookup_is_problem=mvappend(es_lookup_is_problem_small,es_lookup_is_problem_big) | fields - es_lookup_is_problem_small es_lookup_is_problem_big | table es_lookup_*, count_*, *

hettervik · ‎02-14-2022

Thanks, but the format you're suggesting was the first one I tried.

hettervik · ‎02-14-2022

Hi. I'm trying to add a new input with the Tenable add-on: https://splunkbase.splunk.com/app/4060/ When adding a new input I can input a "start time" from when the add-on will start collection data from Tenable, as opposed to "all time" I suppose, but no matter how I format my timestamp, the add-on won't accept it. See screenshot. I've tried all sort of variations, but all fail. I've also looked at the documentation (https://docs.tenable.com/integrations/Splunk/Content/Splunk2/CreateInput.htm) which suggest another time format than the add-on itself (probably not updated), but that isn't working either. Has anyone gotten this to work, and if so, what is the correct way of formating the timestamp?

hettervik · ‎07-01-2021

Did you ever find a solution to this? Seems that we are facing the same issue.

hettervik · ‎06-04-2021

I've installed the Proofpoint TAP SIEM Add-on version 1.3.140, and I'm trying to configure a modular input on my heavy forwarder under "Settings" -> "Data Inputs" -> "Proofpoint TAP SIEM Modular Input". However, after filling out all the required fields and clicking "next", the page just hangs for a while, and then produce the following error message. Encountered the following error while trying to save: Splunkd daemon is not responding: ('Error connecting to /servicesNS/bob/launcher/data/inputs/proofpoint_tap_siem: The read operation timed out',) I also can see the following error messages in the splunkd log. 06-03-2021 16:11:00.622 +0200 WARN HttpListener - Socket error from 127.0.0.1:36448 while accessing /servicesNS/bob/launcher/data/inputs/proofpoint_tap_siem: Broken pipe 06-03-2021 16:11:00.622 +0200 INFO ModularInputs - Argument validation for scheme=proofpoint_tap_siem: script running failed (killed by signal 9: Killed). 06-03-2021 16:11:00.619 +0200 ERROR ModularInputs - Argument validation for scheme=proofpoint_tap_siem: killing process, because executing it took too long (over 30000 msecs). I don't have any other performance problems on my HF, so I don't get why I always get the timeout. For the record I've tried several times, but same error message every time. Have anyone encountered this before and have any tricks for fixing it?

hettervik · ‎06-04-2021

Looks like I'm facing the same issue. When I run the dbxquery directly from the SPL search window it works just fine, but inside the map-command it just hangs forever. Did you ever get this to work?

hettervik · ‎12-16-2020

Hi. Did you find out why you are getting these error messages? We too are getting a lot of errors related to "authentication exception when executing configuration check" and "[HTTP 401] Client is not authenticated" from the configuration_check.py. We can't figure out why.

hettervik · ‎12-11-2020

For me it worked just refreshing the page and trying again. Looks like the server can timeout if it's busy or something. Know I'm late to the party here, but posting it in case someone else is having the same problem.

Posts	277
Solutions	24
Karma Given	117
Karma Received	60
Member Since	‎09-25-2015

Online Status	Offline
Date Last Visited	2 weeks ago

How to find the worst searches in your Splunk envi...

Mastering Splunk Source Types: A Complete Q&A Guid...

How to periodically clean the Splunk ES asset list...

Remove button for AI Assistant in Splunk Enterpris...

Super Optimize your Splunk Stats Searches: Unlocki...

Is there a way to re-index data from an index to a...

How long are asset list field values stored in the...

Nessus security scans dashboard and drilldown

How do Splunk ES create incidents from notable eve...

Why is my lookup field from KV store in datamodel ...

Re: How to stop getting duplicate events from Wind...

Re: RBAC for Notable events?

Re: How can I view the source index where Splunk E...

Re: How to exclude events with same session_id - R...

Re: The Notable Generation doesn't produce the exp...

Re: Is there any way to securely disable the delet...

Is there any way to securely disable the delete co...

Re: How to make the Splunk ES Risk-Based Alerting ...

How to make the Splunk ES Risk-Based Alerting risk...

Re: Getting errors messages Health Check: msg="A s...

Re: Can a universal forwarder run a script? If yes...

How to add setup parameters to a new shell input i...

Re: Can you create/modify a lookup file via REST A...

Re: Timeout when configuring Proofpoint TAP SIEM M...

Re: How to query a lookup table using the REST API

Is there any way to add a custom time to a notable...

How to get the Monitoring Console to correctly ide...

Re: Assets Exceeding Field Limits, Source: [merge]

Re: How to provide a valid start time for the Tena...

How to provide a valid start time for the Tenable ...

Re: Slack Notification Alert Add-on: Error code

Timeout when configuring Proofpoint TAP SIEM Modul...

Re: map search with dbxquery is not returning any ...

Re: Getting errors messages Health Check: msg="A s...

Re: install add-on: Error connecting to /services/...

Join the Conversation