All Posts

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Posts

Hi All, Can someone help me understand which .conf file is responsible to control the connectivity to/from Internet. Wanted to make sure that the so called pure ON Prem Splunk Enterprise solution i... See more...
Hi All, Can someone help me understand which .conf file is responsible to control the connectivity to/from Internet. Wanted to make sure that the so called pure ON Prem Splunk Enterprise solution is unreachable from Internet and most importantly not sending data outside e.g. 1.5 DTI? Thanks in advance.
Hello community, Can anyone please help me understand if the newest vulnerability can exploit a pure on prem Splunk Enterprise clustered solution? Can an arbitrary code be pushed remotely via any m... See more...
Hello community, Can anyone please help me understand if the newest vulnerability can exploit a pure on prem Splunk Enterprise clustered solution? Can an arbitrary code be pushed remotely via any means? Splunk documentation and advisory are not very clear and just saying SE but not mentioning anything about 1.5 DTI and non publicity connected SE instance. Thank you.
You don't appear to be doing anything wrong, given the example you have shared. | makeresults | eval _raw="2023-11-25T21:18:54.244444 [ info ] I am a log message request = GET /api/myendpoi... See more...
You don't appear to be doing anything wrong, given the example you have shared. | makeresults | eval _raw="2023-11-25T21:18:54.244444 [ info ] I am a log message request = GET /api/myendpoint request_id = ff223452" | rex "(?<timestamp>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2})\.\d+\s+\[\s*(?<loglevel>\w+)\s*\]\s+"
I have the following log structure:   2023-11-25T21:18:54.244444  [  info      ]  I am a log message  request = GET /api/myendpoint    request_id = ff223452 I can capture the date and time (without... See more...
I have the following log structure:   2023-11-25T21:18:54.244444  [  info      ]  I am a log message  request = GET /api/myendpoint    request_id = ff223452 I can capture the date and time (without the 244444 part) using: rex field=myfield "(?<timestamp>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2})\.\d+" and timestamp is properly captured. But if I try to extend this and want to capture the log level as well with for example: rex field=myfield "(?<timestamp>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2})\.\d+\s+\[\s*(?<loglevel>\w+)\s*\]\s+" It didn't work; none of the timestamp nor the loglevel is captured. What am I doing wrong?
I often simplify time deltas using stats range(): | stats range(eval(strptime(Time, "%Y-%m-%dT%H:%M:%S.%3N%:z"))) as duration by Employee Country | fieldformat duration=round(duration/86400, 0)."d"... See more...
I often simplify time deltas using stats range(): | stats range(eval(strptime(Time, "%Y-%m-%dT%H:%M:%S.%3N%:z"))) as duration by Employee Country | fieldformat duration=round(duration/86400, 0)."d" ``` or ``` ``` | fieldformat duration=tostring(duration, "duration") ``` ``` for Splunk-normalized [D+]HH:MM:SS display ```
Hi @AL3Z, Windows event log events are stored by Windows in a language-independent format. When using renderXml = true, Splunk does not forward the locale-specific message string. You can further op... See more...
Hi @AL3Z, Windows event log events are stored by Windows in a language-independent format. When using renderXml = true, Splunk does not forward the locale-specific message string. You can further optimize forwarder resource usage by also setting the suppress_* settings to true. In the case of the security event log Microsoft-Windows-Security-Auditing provider/source, event identifier 4688 will have no Message field beginning with "A new process has been created." You must instead use whitelist and blacklist values that reference $XmlRegex and match against the raw XML event. For example: <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"><System><Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-a5ba-3e3b0328c30d}" /><EventID>4688</EventID><Version>2</Version><Level>0</Level><Task>13312</Task><Opcode>0</Opcode><Keywords>0x8020000000000000</Keywords><TimeCreated SystemTime="2023-11-19T16:06:34.0318973Z" /><EventRecordID>139624</EventRecordID><Correlation /><Execution ProcessID="4" ThreadID="344" /><Channel>Security</Channel><Computer>titan</Computer><Security /></System><EventData><Data Name="SubjectUserSid">S-1-5-18</Data><Data Name="SubjectUserName"></Data><Data Name="SubjectDomainName"></Data><Data Name="SubjectLogonId">0x3e7</Data><Data Name="NewProcessId">0x320</Data><Data Name="NewProcessName">C:\Windows\System32\lsass.exe</Data><Data Name="TokenElevationType">%%1936</Data><Data Name="ProcessId">0x1c8</Data><Data Name="CommandLine" /><Data Name="TargetUserSid">S-1-0-0</Data><Data Name="TargetUserName"></Data><Data Name="TargetDomainName"></Data><Data Name="TargetLogonId">0x0</Data><Data Name="ParentProcessName">C:\Windows\System32\wininit.exe</Data><Data Name="MandatoryLabel">S-1-16-16384</Data></EventData></Event> The raw XML event contains a series of <Data> elements, one of which is <Data Name="NewProcessName">. To exclude a specific NewProcessName value, e.g. C:\Windows\System32\lsass.exe, we can construct a blacklist value using the $XmlRegex key. I'll use percent (%) as the regular expression delimiter. You can use one $XmlRegex key to match multiple parts of the raw XML or multiple $XmlRegex keys to make your matches easier to maintain. I've used three $XmlRegex keys to match the Provider, EventID, and Data elements: blacklist3 = $XmlRegex=%<Provider[^>]+Name="Microsoft-Windows-Security-Auditing"% $XmlRegex=%<EventID>4688<\/EventID>% $XmlRegex=%<Data Name="NewProcessName">C:\\Windows\\System32\\lsass\.exe<\/Data>% Note that I've included the provider/source because all Windows events are uniquely identified by a three-tuple of log, provider/source, and event identifier, e.g. Security, Microsoft-Windows-Security-Auditing, and 4688. You can add additional processes to your blacklist by using a regular expression group construct within the NewProcessName match: blacklist3 = $XmlRegex=%<Provider[^>]+Name="Microsoft-Windows-Security-Auditing"% $XmlRegex=%<EventID>4688<\/EventID>% $XmlRegex=%<Data Name="NewProcessName">(C:\\Windows\\System32\\lsass\.exe|C:\\Program Files\\MyApp\\MyProgram\.exe)<\/Data>% This setting would be added to C:\Program Files\SplunkUniversalForwarder\etc\apps\Splunk_TA_windows\local\inputs.conf. Note that the default blacklist1 and blacklist2 values provided by Splunk Add-on for Windows do not work when renderXML = true, so we'll modify those as well. I did not include the provider in the modifications; it's a direct translation of the default. [WinEventLog://Security] disabled = 0 start_from = oldest current_only = 0 evt_resolve_ad_obj = 1 checkpointInterval = 5 renderXml = true suppress_keywords = true suppress_opcode = true suppress_sourcename = true suppress_task = true suppress_text = true suppress_type = true blacklist1 = $XmlRegex=%<EventID>4662<\/EventID>% $XmlRegex=%<Data Name="ObjectType">(?!\s*groupPolicyContainer)% blacklist2 = $XmlRegex=%<EventID>566<\/EventID>% $XmlRegex=%<Data Name="ObjectType">(?!\s*groupPolicyContainer)% blacklist3 = $XmlRegex=%<Provider[^>]+Name="Microsoft-Windows-Security-Auditing"% $XmlRegex=%<EventID>4688<\/EventID>% $XmlRegex=%<Data Name="NewProcessName">(C:\\Windows\\System32\\lsass\.exe|C:\\Program Files\\MyApp\\MyProgram\.exe)<\/Data>% Deploy inputs.conf and restart Splunk Universal Forwarder using your configuration management tool of choice, e.g. a Splunk deployment server.
It would help to know what you've tried already so we don't suggest something that doesn't work. A thing to remember is Splunk can't compare timestamps in text form - they must be converted to epoch... See more...
It would help to know what you've tried already so we don't suggest something that doesn't work. A thing to remember is Splunk can't compare timestamps in text form - they must be converted to epoch (integer) form, first. | makeresults format=csv data="Employee, Time, Country John Doe, 2023-11-15T20:05:31.000+00:00, France John Doe, 2023-11-18T10:00:31.000+00:00, France John Doe, 2023-11-20T10:05:31.000+00:00, United States John Doe, 2023-11-25T20:05:31.000+00:00, United States" ``` Above creates demo data. Delete IRL ``` ``` Convert Time to epoch ``` | eval eTime=strptime(Time, "%Y-%m-%dT%H:%M:%S.%3N%:z") ``` Find the lowest and highest time for each employee/country pair ``` | stats min(eTime) as start, max(eTime) as end by Employee,Country ``` Calculate the duration in days. Add "d" suffix for display ``` | eval duration=round((end-start)/86400,0) . "d" | table Employee duration Country
It will work as it is, but it is poor practice.  Your changes will be lost the next time Splunk_TA_windows is upgraded.
I need help with an employee travel analysis report. I have an index containing information about employee office check-ins in various countries. Events have fields Employee, Time, Country For exam... See more...
I need help with an employee travel analysis report. I have an index containing information about employee office check-ins in various countries. Events have fields Employee, Time, Country For example John Doe, 2023-11-15T20:05:31.000+00:00, France ... John Doe, 2023-11-18T10:00:31.000+00:00, France ... John Doe, 2023-11-20T10:05:31.000+00:00, United States ... John Doe, 2023-11-25T20:05:31.000+00:00, United States   At the end I would like to get the result showing duration in days between first checkin and last checkin per employee per country John Doe, 3d, France John Doe, 5d, United States
@richgalloway  I have made changes to local inputs.conf on this app and deployed it to over 3k servers so we need to move these configurations from local to default to get it work ? Thanks..
Firstly, it's a Cisco-provided app. Vendor-created app are, unfortunately, often sub-par written. They do understand their own products but they often do not understand Splunk well enough. Secondly,... See more...
Firstly, it's a Cisco-provided app. Vendor-created app are, unfortunately, often sub-par written. They do understand their own products but they often do not understand Splunk well enough. Secondly, while you probably could edit app's files, pack it and try to deploy in Cloud, the app would probably not pass appinspect. Thirdly, the description in splunkbase says clearly that it's meant to be installed on a forwarder.
If you have a private link, your Splunk account management team and Splunk support may assist with sizing and configuration; however, I would recommend a heavy forwarder to 1) manage infrastructure a... See more...
If you have a private link, your Splunk account management team and Splunk support may assist with sizing and configuration; however, I would recommend a heavy forwarder to 1) manage infrastructure and transit costs and 2) limit network access to your FMC to devices under your control. The eStreamer client can also be unstable, and having direct access to the heavy forwarder will reduce your MTTR.
    Hi @Arpit-Dwivedi, A description of your call graph may better illustrate the problem, but given your SPL, I would expect the following output, where [] represents a list or array:   R.exp-i... See more...
    Hi @Arpit-Dwivedi, A description of your call graph may better illustrate the problem, but given your SPL, I would expect the following output, where [] represents a list or array:   R.exp-inbound-call R.count L.sapi-outbound-call[] L.count[]   You can often reduce joins to one or more stats commands. This may produce the expected output:   index=sample message.process IN (*app-name1 *app-name2) message.flowName="*| *" | rex field=message.correlationId "(?<UUID>^[0-9a-z-]{0,36})" | eval call-type=case(like('message.process', "%app-name1"), "sapi-outbound-call", like('message.process', "%app-name2"), "exp-inbound-call") | eval {call-type}='message.flowName' | stats list(exp-inbound-call) as exp-inbound-call list(sapi-outbound-call) as sapi-outbound-call by UUID | stats count by exp-inbound-call sapi-outbound-call | stats max(count) as exp-inbound-call-count list(sapi-outbound-call) as sapi-outbound-call list(count) as sapi-outbound-call-count by exp-inbound-call    
Hi Splunkers, I do see 5-6 apps to update in my Splunk cloud, it's asking for restart whenever I'm hovering over update, I don't want to restart it each time, what's the way to update all the apps a... See more...
Hi Splunkers, I do see 5-6 apps to update in my Splunk cloud, it's asking for restart whenever I'm hovering over update, I don't want to restart it each time, what's the way to update all the apps at once and restart once? Thank you! @gcusello 
Hi @onurragacc, Using source as a literal example: index=foo source IN (source1 source2) | table rule1 Explanation | outputlookup rule_lookup the rule_lookup lookup will only contain rows from the... See more...
Hi @onurragacc, Using source as a literal example: index=foo source IN (source1 source2) | table rule1 Explanation | outputlookup rule_lookup the rule_lookup lookup will only contain rows from the search results, both updated and not updated. No additional logic is required. Can you provide an example in SPL with corresponding events and lookup data?
@AL3Z wrote: # DO NOT EDIT THIS FILE! # Please make all changes to files in $SPLUNK_HOME/etc/apps/Splunk_TA_windows/local. # To make changes, copy the section/stanza you want to change from $S... See more...
@AL3Z wrote: # DO NOT EDIT THIS FILE! # Please make all changes to files in $SPLUNK_HOME/etc/apps/Splunk_TA_windows/local. # To make changes, copy the section/stanza you want to change from $SPLUNK_HOME/etc/apps/Splunk_TA_windows/default # into ../local and edit there. Stop right there!  These comments are very important and yet you've chosen to ignore them by editing a file that should not be modified.  What other instructions have you disregarded? The configs shown look good to me, but I am not familiar enough with Windows to know if there's something there that shouldn't be there or vice versa.
0. Please post your searches and such in a preformatted paragraph or a code block. Makes it easier to read. 1. There are no miracles. If repeated search yields different results (and it doesn't cont... See more...
0. Please post your searches and such in a preformatted paragraph or a code block. Makes it easier to read. 1. There are no miracles. If repeated search yields different results (and it doesn't contain any random element), something must be varying across the separate runs - either you're running it over different time windows (for example - "earliest=-1m latest=now" will contain different events depending on when it's run) or your events for given time window change (maybe there are events backfilling the index due to latency problems). Sometimes you might have connectivity problems and not be getting all results from all indexers (but that should throw a warning) or have performance problems and have your searches finalized before they fully complete). 2. You are forcefully overwriting the _time field (honestly, I have no idea why - you could as well just use another field name; if you want it for automatic formatting you could rename it at the very end of your search). 3. As @yuanliu already pointed out - there seems to be a problem with the quality of your data. A process of data onboarding includes finding the proper "main" timestamp within the event (some events can have multiple timestamps; in such case you need to decide which is the primary timestamp for the event) and making sure it's getting parsed out properly so that the event is indexed at the proper point in time That's one of the most important if not the most important part of onboarding the events - you must know where to look for your data. Otherwise you have no way of knowing what data you have, how much data you have, where it is and how to look for it. 4. Yes, latest(X) looks for the latest value of field X. It doesn't mind any other fields. So latest(X) and latest(Y) will show you latest seen values of respectably fields X and Y but they don't have to be from the same event. If one event had only field X, and other one had only field Y, you'd still get both of them in your results since either of them was the last occurrence of respsective field.
TA_for_indexers contains only the installation part needed for indexers (definition of indexes) that are needed for ES to work. But it's just so that ES on its own is "fully installed". Apart from t... See more...
TA_for_indexers contains only the installation part needed for indexers (definition of indexes) that are needed for ES to work. But it's just so that ES on its own is "fully installed". Apart from that Splunk (and ES too) needs to know how to work with specific types of data provided by various kinds of sources. That's what TAs for those sources are for. So yes, if you have 40 _types_ of devices, you might need 40 different TAs. Often TAs contain definitions, parsing rules and CIM-mappings for multiple sources from a single vendor (so you might not need to have a separate TA for every single type of Juniper firewalls, just a single TA able to parse JunOS events).
So if i have 50 devices i need to install the TA on all 50? lets assume cisco, fortinet, palo alto ... So its not enough installing TA on idexers and already such devices are sending the logs to the... See more...
So if i have 50 devices i need to install the TA on all 50? lets assume cisco, fortinet, palo alto ... So its not enough installing TA on idexers and already such devices are sending the logs to the indexer?
Hello, @richgalloway @PickleRick , The regex I used seems effective, but it's unexpectedly blocking all my Windows security events. I've checked the regex, and I haven't specifically blacklisted any... See more...
Hello, @richgalloway @PickleRick , The regex I used seems effective, but it's unexpectedly blocking all my Windows security events. I've checked the regex, and I haven't specifically blacklisted any Windows executables. Could you assist me in analyzing the below list of blacklisted executables? # Copyright (C) 2019 Splunk Inc. All Rights Reserved. # DO NOT EDIT THIS FILE! # Please make all changes to files in $SPLUNK_HOME/etc/apps/Splunk_TA_windows/local. # To make changes, copy the section/stanza you want to change from $SPLUNK_HOME/etc/apps/Splunk_TA_windows/default # into ../local and edit there. # ###### OS Logs ###### [WinEventLog://Security] disabled = 0 start_from = oldest current_only = 0 evt_resolve_ad_obj = 1 checkpointInterval = 5 blacklist1 = EventCode="4662" Message="Object Type:(?!\s*(groupPolicyContainer|computer|user))" blacklist2 = EventCode="5447|4634|5156|4663|4656|5152|5157|4658|4673|4661|4690|4932|4933|5158|4957|5136|4674|4660|4670|5058|5061|4985|4965" blacklist3 = EventCode="4688" Message="(?:New Process Name:).+(?:SplunkUniversalForwarder\\bin\\splunk.exe)|.+(?:SplunkUniversalForwarder\\bin\\splunkd.exe)|.+(?:SplunkUniversalForwarder\\bin\\btool.exe)|.+(?:SplunkUniversalForwarder\\bin\\splunk-powershell.exe)|.+(?:SplunkUniversalForwarder\\bin\\splunk-winprintmon.exe)|.+(?:SplunkUniversalForwarder\\bin\\splunk-regmon.exe)|.+(?:SplunkUniversalForwarder\\bin\\splunk-netmon.exe)|.+(?:SplunkUniversalForwarder\\bin\\splunk-admon.exe)|.+(?:SplunkUniversalForwarder\\bin\\splunk-MonitorNoHandle.exe)|.+(?:SplunkUniversalForwarder\\bin\\splunk-winevtlog.exe)|.+(?:SplunkUniversalForwarder\\bin\\splunk-perfmon.exe)|.+(?:SplunkUniversalForwarder\\bin\\splunkd.exe)|.+(?:SplunkUniversalForwarder\\bin\\splunk-wmi.exe)|.+(?:Windows Defender Advanced Threat Protection\\SenseCncProxy.exe)|.+(?:Windows Defender Advanced Threat Protection\\SenseCM.exe)|.+(?:Windows Defender Advanced Threat Protection\\MsSense.exe)|.+(?:Microsoft\\Windows Defender\\Platform\\.*\MsMpEng.exe)|.+(?:Microsoft\\Windows Defender\\Platform\\.*\\MpCmdRun.exe)|.+(?:Microsoft\\Windows Defender Advanced Threat Protection\\Platform\\.*\\MsSense.exe)|.+(?:Microsoft\\Windows Defender\\Platform\\.*\\MsMpEng.exe)|.+(?:Microsoft\\Windows Defender Advanced Threat Protection\Platform\.*\\SenseIR.exe)|.+(?:Microsoft\\Windows Defender Advanced Threat Protection\\DataCollection\\.*\\OpenHandleCollector.exe)|.+(?:ForeScout SecureConnector\\SecureConnector.exe)|.+(?:Windows Defender Advanced Threat Protection\\SenseIR.exe)|.+(?:Rapid7\\Insight Agent\\components\\insight_agent\\.*\\get_proxy.exe)|.+(?:Rapid7\\Insight Agent\\components\\insight_agent\\.*\\ir_agent.exe|.+(?:Tanium\\Tanium Client\\TaniumCX.exe)|.+(?:AzureConnectedMachineAgent\\GCArcService\\GC\\gc_worker.exe)|.+(?:AzureConnectedMachineAgent\\GCArcService\\GC\\gc_service.exe)|.+(?:WindowsPowerShell\\Modules\\gytpol\\Client\\fw.*\\GytpolClientFW.*.exe)|.+(?:AzureConnectedMachineAgent\\azcmagent.exe)|.+(?:Microsoft Monitoring Agent\\Agent\\MonitoringHost.exe)" blacklist4 = EventCode="4688" Message="(?:New Process Name:).+(?:Tanium\\Tanium Client)" blacklist5 = EventCode="4688" Message="(?:Creator Process Name:).+(?:Tanium\\Tanium Client)" renderXml=true index = es_winsec Thanks...