All Posts

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Posts

There can be many things done with data within Splunk ingestion process so the general "how to find XXX with one search" question response should usually be appended with a big warning "if your case ... See more...
There can be many things done with data within Splunk ingestion process so the general "how to find XXX with one search" question response should usually be appended with a big warning "if your case is somewhat unusual, this might not work at all". I think I could even write props/transforms to rewrite each event's source to a random value (and route to a random index ). Try finding where such event came from unless you _know_ it.
Just to clear things up—I’m not the same person, just a friend who happens to share the same frustration. We’re both in the trenches, managing Splunk forwarders on Linux, and trust me, it’s a shared ... See more...
Just to clear things up—I’m not the same person, just a friend who happens to share the same frustration. We’re both in the trenches, managing Splunk forwarders on Linux, and trust me, it’s a shared experience we can all relate to. We're not asking for much, just some consistency and automation in how we manage our installs. A proper YUM repo would save us from the ongoing dance of manual installs and updates, and help us maintain the stability and security that every sysadmin craves. So no, I’m not the one who’s been posting under multiple accounts, but I am definitely right there with them, dealing with the same pain points. Let’s hope we can get some traction on this, and maybe one day, our dream of a YUM repo will be more than just a wish. Cheers, A Fellow Sysadmin Who Gets the Struggle
That's a valid point about HEC events being able to specify any source value, however, that would not be associated with an app and so would not apply to the OP's request. It's possible a transform ... See more...
That's a valid point about HEC events being able to specify any source value, however, that would not be associated with an app and so would not apply to the OP's request. It's possible a transform can be used to change the source name to something else.  Those can be found by searching the transforms.conf file on an indexer (or on the Cluster Manager).
This way you can only (if the regex matches) extract indexed field, not modify the original event (maybe except when you overwrite the _raw event). You're looking for the SEDCMD functionality. I'd a... See more...
This way you can only (if the regex matches) extract indexed field, not modify the original event (maybe except when you overwrite the _raw event). You're looking for the SEDCMD functionality. I'd also slightly modify your regex since you're looking for a base64-encoded contents which may not contain neither backslash nor a quote. SEDCMD-trim-file = s/(\\"file\\":\s*\\")([^\\"]{5000,}?)/\1long_file/g  See it here https://regex101.com/r/8nX7FY/1 (the regex101 substitution uses a bit different format to SEDCMD - it uses $1 instead of \1)
I just created a feature request on ideas.splunk.com: Provide a YUM repository for Splunk Feel free to vote it up
Maybe not you, but there were at least two separate freshly created accounts with just one comment in this thread in their history, created within some 10 minutes of each other and posting a very sim... See more...
Maybe not you, but there were at least two separate freshly created accounts with just one comment in this thread in their history, created within some 10 minutes of each other and posting a very similarily built comments. Accident? I doubt it. The ideas portal has been alive for at least a few years now.
The REGEX does not match the sample data because backslashes must be escaped.  Try REGEX = \\"file\\":\s*\\"(.{5000,}?),"
That's only half of the problem. The other half is that it can as well be completely externally submitted data - with HEC input you can set the source to anything you want (actually even for monitor... See more...
That's only half of the problem. The other half is that it can as well be completely externally submitted data - with HEC input you can set the source to anything you want (actually even for monitor inputs you could set the source to anything you want but it's rather unlikely that someone deliberately rewrites the source field to mislead you as to the real origin of the data) but stil have one HEC source.
Whoa there, Sherlock! Let’s not assume every account here is just me in a trench coat and a fake mustache. Some of us have been lurking in the shadows, waiting for the perfect moment to echo the exac... See more...
Whoa there, Sherlock! Let’s not assume every account here is just me in a trench coat and a fake mustache. Some of us have been lurking in the shadows, waiting for the perfect moment to echo the exact same frustrations in slightly different words. Totally different people, promise. But hey, thanks for pointing out ideas.splunk.com! After only 13 years, it’s nice to know there’s an official wishing well where dreams of a YUM repo can go to...um, simmer? Who knows, maybe we’ll have a repo by the time this thread can vote or rent a car. Appreciate the tip!
Hi How can I check the cherrypy version for Splunk 7.3.8? There are no cherrypy related files in splunk/share/3rdparty. Thank you.
Since it's inputs.conf, they'll be distributed among the forwarders and so harder to search.  The best approach likely will be to use the CLI to scan the deployment-apps directory on the Deployment S... See more...
Since it's inputs.conf, they'll be distributed among the forwarders and so harder to search.  The best approach likely will be to use the CLI to scan the deployment-apps directory on the Deployment Server. find /opt/splunk/etc/deployment-apps -name inputs.conf -print0 | xargs -r0 grep -i "logs\/abc\/def" A complicating factor is the potential use of wildcards ('*' and "...") in monitor stanzas.  I'm sure you can modify the grep to find those.
Hi, I have a log source (/logs/abc/def). I want to know what are the apps  using this log source in their inputs.conf.   Can someone provide me the search query?
Hi @ITWhisperer, Thank you for your feedback. The Regex works, but according to @PickleRick I will need to adjust my approach. Kind regards, Dan
Hi @PickleRick, Thank you for your valuable feedback. Index-Time Extractions: You're right that named capture groups might not be supported at index time. I'll modify my configurations to use num... See more...
Hi @PickleRick, Thank you for your valuable feedback. Index-Time Extractions: You're right that named capture groups might not be supported at index time. I'll modify my configurations to use numbered capture groups to ensure they function correctly. Rewriting _raw: I appreciate you highlighting the potential issues with rewriting _raw to contain key-value pairs. My intention was to reduce the size of the events by removing unnecessary data, but I see how this could lead to unintended side effects during indexing. I'll reconsider this approach. Structured Data Parsing: Your point about the risks of using regex to parse XML is well-taken. Given that XML fields may vary in order and presence, relying on regex could indeed cause problems. Utilizing Splunk's structured data parsing capabilities seems like a better solution. Next steps: To achieve my goal of reducing the indexed data volume for EventID=4627 events, I'd like to leverage Splunk's XML parsing features. Specifically, I'm thinking of using INDEXED_EXTRACTIONS = xml and configuring EXCLUDE rules in props.conf to omit the unwanted fields at index time. Example Configuration BEFORE: [reduce_event_raw] REGEX = (?ms)<Event[^>]*>.*?<System>.*?<EventID>4627<\/EventID>.*?<Computer>(?<Computer>[^<]*)<\/Computer>.*?<Data\s+Name='SubjectUserName'>(?<SubjectUserName>[^<]*)<\/Data>.*?<Data\s+Name='TargetUserName'>(?<TargetUserName>[^<]*)<\/Data>.*?<Data\s+Name='LogonType'>(?<LogonType>[^<]*)<\/Data> FORMAT = Computer::$1 SubjectUserName::$2 TargetUserName::$3 LogonType::$4 DEST_KEY = _raw Example Configuration AFTER: [XmlWinEventLog:Security] INDEXED_EXTRACTIONS = xml KV_MODE = none EXCLUDE = (?i)(SubjectUserSid|SubjectDomainName|SubjectLogonId|TargetUserSid|TargetDomainName|TargetLogonId|EventIdx|EventCountTotal|GroupMembership) Do you think this approach would effectively remove the unnecessary fields before indexing while maintaining reliable field extraction for the essential data? If you have any suggestions or best practices for this method, I'd greatly appreciate your guidance. Regards, Dan
DAS = Datamodel Accelerated Summaries. In case of metrics it shouldn't apply. I'm not sure if you even can do that against metric indexes. Anyway, does the current state reported by the rest command... See more...
DAS = Datamodel Accelerated Summaries. In case of metrics it shouldn't apply. I'm not sure if you even can do that against metric indexes. Anyway, does the current state reported by the rest command or the settings->indexes (in terms of current usage, not the settings) correspond to the data size on the disk?
it's a Fortianalyzer via a custom TCP port.  Probably the simplest solution will be configuring a new log forwarding directly on FAZ with filtering. Thanks for the help!
Hi Are you trying to collect macOS logs or Windows logs? If you are trying those from macOS, there are logd input method which you could try. Unfortunately there is some issues with current splunk ... See more...
Hi Are you trying to collect macOS logs or Windows logs? If you are trying those from macOS, there are logd input method which you could try. Unfortunately there is some issues with current splunk versions with it (see https://community.splunk.com/t5/Getting-Data-In/Wrong-parameters-on-macOS-and-logd-input/td-p/702261). Until splunk fix this you must use e.g. TA for nix or use your own scripts to use "log show" command with correct parameters. r. Ismo
@PickleRick  Thanks again. After reading again your past message and checking with my team, the index saturating in question is a default one so i updated my post with all informations i could give... See more...
@PickleRick  Thanks again. After reading again your past message and checking with my team, the index saturating in question is a default one so i updated my post with all informations i could give -> _metrics. 1) Checked, no others than _metrics are pointing on the $SPLUNK_DB/_metrics/ directory 2) Warm and Hot buckets only. What are DAS data ? I don't know about what you said last, but that might have cause the issue.
If your FW sends those logs via syslog feed, then it's probably easier to add e.g. rsyslog where those are sent and do filtering/forwarding there instead of use splunk transforms.conf for that?
You should also remember that Splunk has this as a default  splunk btool indexes list volume:_splunk_summaries|egrep '(\[volume|path)' [volume:_splunk_summaries] path = $SPLUNK_DB