All Posts

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Posts

Hi @ITWhisperer, Thank you for your feedback. The Regex works, but according to @PickleRick I will need to adjust my approach. Kind regards, Dan
Hi @PickleRick, Thank you for your valuable feedback. Index-Time Extractions: You're right that named capture groups might not be supported at index time. I'll modify my configurations to use num... See more...
Hi @PickleRick, Thank you for your valuable feedback. Index-Time Extractions: You're right that named capture groups might not be supported at index time. I'll modify my configurations to use numbered capture groups to ensure they function correctly. Rewriting _raw: I appreciate you highlighting the potential issues with rewriting _raw to contain key-value pairs. My intention was to reduce the size of the events by removing unnecessary data, but I see how this could lead to unintended side effects during indexing. I'll reconsider this approach. Structured Data Parsing: Your point about the risks of using regex to parse XML is well-taken. Given that XML fields may vary in order and presence, relying on regex could indeed cause problems. Utilizing Splunk's structured data parsing capabilities seems like a better solution. Next steps: To achieve my goal of reducing the indexed data volume for EventID=4627 events, I'd like to leverage Splunk's XML parsing features. Specifically, I'm thinking of using INDEXED_EXTRACTIONS = xml and configuring EXCLUDE rules in props.conf to omit the unwanted fields at index time. Example Configuration BEFORE: [reduce_event_raw] REGEX = (?ms)<Event[^>]*>.*?<System>.*?<EventID>4627<\/EventID>.*?<Computer>(?<Computer>[^<]*)<\/Computer>.*?<Data\s+Name='SubjectUserName'>(?<SubjectUserName>[^<]*)<\/Data>.*?<Data\s+Name='TargetUserName'>(?<TargetUserName>[^<]*)<\/Data>.*?<Data\s+Name='LogonType'>(?<LogonType>[^<]*)<\/Data> FORMAT = Computer::$1 SubjectUserName::$2 TargetUserName::$3 LogonType::$4 DEST_KEY = _raw Example Configuration AFTER: [XmlWinEventLog:Security] INDEXED_EXTRACTIONS = xml KV_MODE = none EXCLUDE = (?i)(SubjectUserSid|SubjectDomainName|SubjectLogonId|TargetUserSid|TargetDomainName|TargetLogonId|EventIdx|EventCountTotal|GroupMembership) Do you think this approach would effectively remove the unnecessary fields before indexing while maintaining reliable field extraction for the essential data? If you have any suggestions or best practices for this method, I'd greatly appreciate your guidance. Regards, Dan
DAS = Datamodel Accelerated Summaries. In case of metrics it shouldn't apply. I'm not sure if you even can do that against metric indexes. Anyway, does the current state reported by the rest command... See more...
DAS = Datamodel Accelerated Summaries. In case of metrics it shouldn't apply. I'm not sure if you even can do that against metric indexes. Anyway, does the current state reported by the rest command or the settings->indexes (in terms of current usage, not the settings) correspond to the data size on the disk?
it's a Fortianalyzer via a custom TCP port.  Probably the simplest solution will be configuring a new log forwarding directly on FAZ with filtering. Thanks for the help!
Hi Are you trying to collect macOS logs or Windows logs? If you are trying those from macOS, there are logd input method which you could try. Unfortunately there is some issues with current splunk ... See more...
Hi Are you trying to collect macOS logs or Windows logs? If you are trying those from macOS, there are logd input method which you could try. Unfortunately there is some issues with current splunk versions with it (see https://community.splunk.com/t5/Getting-Data-In/Wrong-parameters-on-macOS-and-logd-input/td-p/702261). Until splunk fix this you must use e.g. TA for nix or use your own scripts to use "log show" command with correct parameters. r. Ismo
@PickleRick  Thanks again. After reading again your past message and checking with my team, the index saturating in question is a default one so i updated my post with all informations i could give... See more...
@PickleRick  Thanks again. After reading again your past message and checking with my team, the index saturating in question is a default one so i updated my post with all informations i could give -> _metrics. 1) Checked, no others than _metrics are pointing on the $SPLUNK_DB/_metrics/ directory 2) Warm and Hot buckets only. What are DAS data ? I don't know about what you said last, but that might have cause the issue.
If your FW sends those logs via syslog feed, then it's probably easier to add e.g. rsyslog where those are sent and do filtering/forwarding there instead of use splunk transforms.conf for that?
You should also remember that Splunk has this as a default  splunk btool indexes list volume:_splunk_summaries|egrep '(\[volume|path)' [volume:_splunk_summaries] path = $SPLUNK_DB
There is probably another way but the easiest way around it is to set up an intermediate forwarder (a UF so that it doesn't do any parsing, filtering and whatnot) with one input and two outputs and j... See more...
There is probably another way but the easiest way around it is to set up an intermediate forwarder (a UF so that it doesn't do any parsing, filtering and whatnot) with one input and two outputs and just send both to Cloud as well as to your HF from this "cloud all" environment.
Ouch. 1. If you're using numbered capture groups you don't have to name them. (I'm not even sure if index-time extractions support named capture groups). 2. Assuming your regex was right you'd get ... See more...
Ouch. 1. If you're using numbered capture groups you don't have to name them. (I'm not even sure if index-time extractions support named capture groups). 2. Assuming your regex was right you'd get a key::value pairs in your raw event. Are you sure that's what you want? Also, this will cause "interesting" side effects since that data would get split into terms at major breakers and would get indexed as indexed fields. 3. Manipulating structured data with regexes is asking for trouble. You have no guarantee that the fields will always be in the same order (and they might not always contain full data). That's why you use structured data format.
i have events that contains a specific field that sometimes contain a very long field which make the rest of the event be truncated, i want to remove this field or change it "long field detected". t... See more...
i have events that contains a specific field that sometimes contain a very long field which make the rest of the event be truncated, i want to remove this field or change it "long field detected". the problematic field call "file" and i should catch it's last appearnce, also i want the data after it so i should stop the removal after the first "," (comma). also the event contains nested fields. i've tried props.conf+transform conf like that:ete but it doesn't work. here is an example for 1 event: deleted due to security reasons 
Hi @corti77 , I don't use SC4S but usually rsyslog and a Universal Forwarder that's the same thing. So I usually (except when there are very big files to read)use the batch command, instead of moni... See more...
Hi @corti77 , I don't use SC4S but usually rsyslog and a Universal Forwarder that's the same thing. So I usually (except when there are very big files to read)use the batch command, instead of monitor command in the inputs.conf. Ciao. Giuseppe
Hi @daniel99 , did you installed the Splunk_TA_Windows ( https://splunkbase.splunk.com/app/742 ) on your Splunk? Ciao. Giuseppe  
The /raw endpoint should not need the ?auto_extract_timestamp=true parameter.
OK. Instead of creating new accounts just to post the same content which is completely pointless, the thing you (and everyone who finds this idea important) can do is log into https://ideas.splunk.co... See more...
OK. Instead of creating new accounts just to post the same content which is completely pointless, the thing you (and everyone who finds this idea important) can do is log into https://ideas.splunk.com and create or upvote a relevant idea there. If it gathers enough visibility it might get considered. Just posting random rants here won't accomplish much.
Hi, I wonder the easiest way to monitor the deletion of files/folders in a CIFS netapp using splunk. I saw an Add-on available, could someone share any experience with this use case? I have a SC4S... See more...
Hi, I wonder the easiest way to monitor the deletion of files/folders in a CIFS netapp using splunk. I saw an Add-on available, could someone share any experience with this use case? I have a SC4S in place so I thought to configure syslog in NetApp to be sent to SC4S and start digging into the logs. Is there any App I could leverage to ease the pain? many thanks  
OK. So that is interesting. I'd check then 1) If there isn't by any chance another definition pointing to the same directory (for example one index defined by means of $SPLUNK_DB and another based o... See more...
OK. So that is interesting. I'd check then 1) If there isn't by any chance another definition pointing to the same directory (for example one index defined by means of $SPLUNK_DB and another based on volume) 2) What actually consumes the disk in this directory. Just the buckets or something else? Maybe you have a lot of DAS data. Or maybe you're ingesting a lot of data with indexed extractions and have bloated idx files...
Dear Splunk, Adding my voice here, because honestly, how is this still a thing? It’s like watching a toddler grow up but refusing to wear shoes because ‘barefoot builds character.’ We’re not trying ... See more...
Dear Splunk, Adding my voice here, because honestly, how is this still a thing? It’s like watching a toddler grow up but refusing to wear shoes because ‘barefoot builds character.’ We’re not trying to strip you of your rugged charm—we’re just asking you to stop tracking mud into the data center. Look, it’s not just about convenience. A proper YUM repo means: Consistency: No more “Did we grab the right version from the website?” anxiety. Efficiency: Automation beats playing 'Where’s the Download Link?' every release. Security: Signed RPMs and authenticated repos mean we sleep better at night. (And you don’t want to mess with my sleep.) You’re a billion-dollar company, not a weekend side project. If Bob’s Discount Monitoring Software has a YUM repo, so can you. Let’s not make this a 14th-birthday discussion, or worse, a sweet sixteen. Yours in exasperation, A Sysadmin Who Just Wants to Automate
Try something like this (?ms)<Event[^>]*>.*?<System>.*?<Provider\s+Name='(?<ProviderName>[^']*)'\s+Guid='(?<ProviderGuid>[^']*)'.*?<EventID>(?<EventID>\d+)<\/EventID>.*?<Version>(?<Version>\d+)<\/Ve... See more...
Try something like this (?ms)<Event[^>]*>.*?<System>.*?<Provider\s+Name='(?<ProviderName>[^']*)'\s+Guid='(?<ProviderGuid>[^']*)'.*?<EventID>(?<EventID>\d+)<\/EventID>.*?<Version>(?<Version>\d+)<\/Version>.*?<Level>(?<Level>\d+)<\/Level>.*?<Task>(?<Task>\d+)<\/Task>.*?<Opcode>(?<Opcode>\d+)<\/Opcode>.*?<Keywords>(?<Keywords>[^<]*)<\/Keywords>.*?<TimeCreated\s+SystemTime='(?<SystemTime>[^']*)'.*?<EventRecordID>(?<EventRecordID>\d+)<\/EventRecordID>.*?<Correlation\s+ActivityID='(?<ActivityID>[^']*)'.*?<Execution\s+ProcessID='(?<ProcessID>\d+)'\s+ThreadID='(?<ThreadID>\d+)'.*?<Channel>(?<Channel>[^<]*)<\/Channel>.*?<Computer>(?<Computer>[^<]*)<\/Computer>.*?<EventData>.*?<Data\s+Name='SubjectUserSid'>(?<SubjectUserSid>[^<]*)<\/Data>.*?<Data\s+Name='SubjectUserName'>(?<SubjectUserName>[^<]*)<\/Data>.*?<Data\s+Name='SubjectDomainName'>(?<SubjectDomainName>[^<]*)<\/Data>.*?<Data\s+Name='SubjectLogonId'>(?<SubjectLogonId>[^<]*)<\/Data>.*?<Data\s+Name='TargetUserSid'>(?<TargetUserSid>[^<]*)<\/Data>.*?<Data\s+Name='TargetUserName'>(?<TargetUserName>[^<]*)<\/Data>.*?<Data\s+Name='TargetDomainName'>(?<TargetDomainName>[^<]*)<\/Data>.*?<Data\s+Name='TargetLogonId'>(?<TargetLogonId>[^<]*)<\/Data>.*?<Data\s+Name='LogonType'>(?<LogonType>[^<]*)<\/Data>.*?<Data\s+Name='EventIdx'>(?<EventIdx>[^<]*)<\/Data>.*?<Data\s+Name='EventCountTotal'>(?<EventCountTotal>[^<]*)<\/Data>.*?<Data\s+Name='GroupMembership'>(?<GroupMembership>.*?)<\/Data>.*?<\/EventData>.*?<\/Event> https://regex101.com/r/19eJtB/1  
Hello everyone, I'm facing a persistent issue with executing a script via a playbook in Splunk SOAR that uses WinRM. Here's the context: I've created a playbook that is supposed to isolate a host v... See more...
Hello everyone, I'm facing a persistent issue with executing a script via a playbook in Splunk SOAR that uses WinRM. Here's the context: I've created a playbook that is supposed to isolate a host via WinRM. The script works perfectly when I run it manually using the "Run Script" action from Splunk SOAR: the host gets isolated. However, when the same script is executed by the playbook, the execution is marked as "successful," but none of the expected outcomes occur: the host is not isolated. To be more precise: I added an elevation check in the script, which relaunches in administrator mode with -Verb RunAs if necessary. This works perfectly for the manual action. The script writes to a log file located in C:\Users\Public\Documents to avoid permission issues, but the log file is not created when executed by the playbook. I've tried other directories and even simplified the logic to just disable a network adapter with Disable-NetAdapter, but nothing seems to work. In summary, everything works fine when done manually, but not when automated via the playbook. I have the impression that there's a difference in context between manual execution and playbook execution that's causing the issue, perhaps related to permissions or WinRM session restrictions. Does anyone have any idea what might be preventing the playbook from executing this script correctly, or any suggestions for workarounds? I'm really running out of ideas and any help would be greatly appreciated. Thanks in advance!