All Posts

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Posts

Sorry, I've opened a new post about my problem. I think that I have given some wrong information here, which I have noticed in the meantime. https://community.splunk.com/t5/Getting-Data-In/Collect-j... See more...
Sorry, I've opened a new post about my problem. I think that I have given some wrong information here, which I have noticed in the meantime. https://community.splunk.com/t5/Getting-Data-In/Collect-journalctl-events-with-a-Splunk-UF-to-Cribl-Stream-in/m-p/689510#M114765  
Hello, Here I have a small picture of how the environment is structured: Red arrow -> Source Splunk TCP (Cribl Stream)   I'm trying to forward the journald data from the Splunk Universal Forw... See more...
Hello, Here I have a small picture of how the environment is structured: Red arrow -> Source Splunk TCP (Cribl Stream)   I'm trying to forward the journald data from the Splunk Universal Forwarder to the Cribl Worker (Black to blue box). I have configured the forwarding of the journald data using the instructions from Splunk. (Get data with the Journald input - Splunk Documentation)   I can forward the journald data and it also arrives at the cribl worker. Problem: the cribl worker cannot distinguish the individual events from the journald data or does not know when a single event is over and thus combines several individual events into one large one. The Cribl Worker always merges about 5-8 journald events. (I have marked the individual events here. However, they arrive as such a block, sometimes more together, sometimes less.) Event 1: Invalid user test from 111.222.333.444port 1111pam_unix(sshd:auth):check pass; userunknownpam_unix(sshd:auth):authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=111.222.333.444Failed password forinvalid user testfrom 111.222.333.444 port1111 ssh2error: Received disconnect from 111.222.333.444port 1111:13: Unableto authenticate [preauth]Disconnected from invaliduser test 111.222.333.444port 1111 [preauth]   What I tested: If I have the journald data from the universal forwarder not forwarded via a cribl worker, but via a heavy forwarder (The blue box in the picture above is then no longer a Cribl Worker but a Splunk Heavy Forwarder), then the events are individual and easy to read. Like this: Event 1:   Invalid user testfrom 111.222.333.444 port1111   Event 2:   pam_unix(sshd:auth):check pass; userunknown   Event 3:   pam_unix(sshd:auth):authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=111.222.333.444   Event 4:   Failed password forinvalid user testfrom 111.222.333.444 port1111 ssh2   Event 5:   error: Received disconnectfrom 111.222.333.444 port1111:13: Unable toauthenticate [preauth]   Event 6:   Disconnected from invaliduser test 111.222.333.444port 1111 [preauth]   -------------------------------- I'm looking for a solution that I can send the journald data as shown in the figure above, but the journald data will be sent as in the second case. Thanks in advance for your help.
Extending @ITWhisperer 's answer - unless you have a third-party solution (some form of asset inventory software or even your own scripted input listing installed software), Splunk on its own cannot ... See more...
Extending @ITWhisperer 's answer - unless you have a third-party solution (some form of asset inventory software or even your own scripted input listing installed software), Splunk on its own cannot tell you since it only works on the data you give it. So by default you can only pull what your Windows machine produces (event logs, maybe some log files). So if you can find this info in what Windows report on its own - good, you can use it. But I don't recall that it does.
Having said that - as with most of the questions starting with "how to find all" - it's possible to do it only for a specific subset of cases. There are ways of creating the searches so that you won'... See more...
Having said that - as with most of the questions starting with "how to find all" - it's possible to do it only for a specific subset of cases. There are ways of creating the searches so that you won't know what they're using effectively for searching (aliases, eventtypes, tags, subsearches, lookups...).
This is obviously a mistake on the docs page (unfortunately dev docs don't include the feedback form). How would you write JS code with Python SDK? It makes no sense.
There's a portal for such feature requests - https://ideas.splunk.com/  
Where did you put your props.conf? (on which component) And what does your ingest process look like? Because that's apparently not data from a windows eventlog input.
What are you using for authentication? If you are using external authentication source (like LDAP or SAML) your users will get re-created as soon as they authenticate using that source.
thanks @KendallW - I think $result.field$ will not work in this scenario? I am already using he subject line as you mentioned but it is having a blank value (in Email I receive) for the variables. 
This is so vague - there can be a gazillion of reasons for you not being able to connect. Does your desktop meet the minimum parameters for Splunk installation? Is the splunkd process running? Do yo... See more...
This is so vague - there can be a gazillion of reasons for you not being able to connect. Does your desktop meet the minimum parameters for Splunk installation? Is the splunkd process running? Do you see errors in $SPLUNK_HOME/var/log/splunk/splunkd.log? Did you change anything in your computer's configuration (most importantly network/firewall settings)?
It doesn't work like that. For TA_auditd to work you ingest contents of /var/log/audit/auditd.log in text form. The settings you're trying to manipulate do completely different things - they tell Sp... See more...
It doesn't work like that. For TA_auditd to work you ingest contents of /var/log/audit/auditd.log in text form. The settings you're trying to manipulate do completely different things - they tell Splunk how to _interpret_ the received data. You can't use them to make json from plain text or something like that.
please how do i hashout as proposed by the solution  
This is a 7 years old thread. You'd get much more visibility if you posted your question as a new thread (possibly dropping in a link to this thread for reference if it's relevant to your case).
I really appreciate your advices, Thank you for discussion
Sure. Whatever rocks your boat But seriously - it's like ITIL - adopt and adapt. If something works for you and you are aware of your approach's limitations - go ahead.  
For this situation, we have a weekly alert that shows "missing hosts" | tstats latest(_time) as latest where NOT index=main AND NOT index="*-summary" earliest=-30d by index, host | eval DeltaSeconds... See more...
For this situation, we have a weekly alert that shows "missing hosts" | tstats latest(_time) as latest where NOT index=main AND NOT index="*-summary" earliest=-30d by index, host | eval DeltaSeconds = now() - latest | where DeltaSeconds>604800 | eval LastEventTime = strftime(latest,"%Y-%m-%d %H:%M:%S") | eval DeltaHours = round(DeltaSeconds/3600) | eval DeltaDays = round(DeltaHours/24) | join index [| inputlookup generated_file_with_admins_mails.csv] | table index, host, LastEventTime, DeltaHours, DeltaDays, email_to Using the sendresults app, this Splunk alerts the responsible employee(s) about these hosts. Now this search shows only hosts that haven't sent Syslog for more than 7 days and that's OK for us In most cases, this alert shows only hosts that we removed from our infrastructure But if it will be necessary, I can run this alert more frequently or separate it into several searches with different "missing" conditions I understand that this approach cannot handle, for example, some intermittent network or software lags, but I have used this approach for about a year and all is quite fine, excluding some rare cases (like this topic)
While the general question is of course valid and needs to be considered properly, I saw similar cases in my experience - splitting data from a single source into separate indexes. The most typical ... See more...
While the general question is of course valid and needs to be considered properly, I saw similar cases in my experience - splitting data from a single source into separate indexes. The most typical case is when you have a single solution providing logs for separate business entities (like a central security appliance protecting multiple divisions or even companies from a single business group). You might want to split events so that each unit has access only to its own events (possibly with some overseeing security team having access to all those indexes). So there are valid use cases for similar setups
Hello richgalloway I've checked for the splunkd.log for the past 1 week - no errors found. 8080 is closed even on the old clusters but we never had troubles with Replication and Search Factor.  98... See more...
Hello richgalloway I've checked for the splunkd.log for the past 1 week - no errors found. 8080 is closed even on the old clusters but we never had troubles with Replication and Search Factor.  9887 and 8089 ports are all open across all the clusters. But still the fixup tasks pending - 301 Fixup tasks - In progress - 0
@anooshac  Can you please share your full sample code? KV
Your data illustration strongly suggest that it is part of a JSON event like,     {"message":"sypher:[tokenized] build successful -\xxxxy {\"data\":{\"account_id\":\"ABC123XYZ\",\"activity\":{\"ti... See more...
Your data illustration strongly suggest that it is part of a JSON event like,     {"message":"sypher:[tokenized] build successful -\xxxxy {\"data\":{\"account_id\":\"ABC123XYZ\",\"activity\":{\"time\":\"2024-05-31T12:37:25Z\"}}", "some_field":"somevalue", "some_other_field": "morevalue"}     In this case, Splunk should have given you a field named "message"  that has this value:      "message":"sypher:[tokenized] build successful -\xxxxy {\"data\":{\"account_id\":\"ABC123XYZ\",\"activity\":{\"time\":\"2024-05-31T12:37:25Z\"}}"     What the developer is trying to do is to embed more data in this field, partially also in JSON.  For long-term maintainability, it is best not to treat that as text, either.  This means that regex is not the right tool for the job.  Instead,  try to get the embedded JSON first. There is just one problem (in addition to missing a closing double quote for the time value): the string \xxxxy is illegal in JSON.  If this is the real data, Splunk would have bailed and NOT give you a field named "message".  In that case, you will have to deal with that first.  Let's explore how later. For now, suppose your data is actually   {"message":"sypher:[tokenized] build successful -\\\xxxxy {\"data\":{\"account_id\":\"ABC123XYZ\",\"activity\":{\"time\":\"2024-05-31T12:37:25Z\"}}", "some_field":"somevalue", "some_other_field": "morevalue"}   As such, Splunk would have given you a value for message like this:   sypher:[tokenized] build successful -\xxxxy {"data":{"account_id":"ABC123XYZ","activity":{"time":"2024-05-31T12:37:25Z"}}   Consequently, all you need to do is   | eval jmessage = replace(message, "^[^{]+", "") | spath input=jmessage   You will get the following fields data.account_id data.activity.time some_field some_other_field ABC123XYZ 2024-05-31T12:37:25Z somevalue morevalue Here is an emulation of the "correct" data you can play with and compare with real data   | makeresults | eval _raw = "{\"message\":\"sypher:[tokenized] build successful -\\\xxxxy {\\\"data\\\":{\\\"account_id\\\":\\\"ABC123XYZ\\\",\\\"activity\\\":{\\\"time\\\":\\\"2024-05-31T12:37:25Z\\\"}}\", \"some_field\":\"somevalue\", \"some_other_field\": \"morevalue\"}" | spath ``` data emulation above ```   Now, if your raw data indeed contains \xxxxy inside a JSON block, you can still rectify that with text manipulation so you get a legal JSON.  But you have to tell your developer that they are logging bad JSON. (Recently there was a case where an IBM mainframe plugin sent Splunk bad data like this.  It is best for the developer to fix this kind of problem.)