All Posts

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Posts

Typically that's a result of wrong scope or insufficient access - your lookup is either private or exported only to the app you've created it in but you're searching from another app (typically the s... See more...
Typically that's a result of wrong scope or insufficient access - your lookup is either private or exported only to the app you've created it in but you're searching from another app (typically the search app)
@ITWhisperer and @livehybrid . Both responses helped me understand the overall issue and I thank you both.   Another method that I worked on is to use 2 Regex expressions in props.conf: Regex 1 FA... See more...
@ITWhisperer and @livehybrid . Both responses helped me understand the overall issue and I thank you both.   Another method that I worked on is to use 2 Regex expressions in props.conf: Regex 1 FAILED.+\:\s(?<LogFile>.+)(\n)(?<Reason1>.+(\n).+)          - that grabs "Host key verification failed lost connection" OR "You are attempting to access a system owned by XYZ" into the Reason1 field The second Regex:  Agreement\sfor\sdetails\.(\n)(?<Reason2>.+) That grabs: "scp: /logs/rsyslog/server02/: Not a directory" into the Reason2 field In the search there is a case statement to make it work | eval Message=case(like(Reason1,"%You are%"),Reason2,1==1,Reason1) It sounds a bit inefficient, but it is working for the report. Thank you both again.
Splunk is not Excel But seriously. For Splunk every result row is... well, a separate row. Depending on the actual use case you could cheat a bit but the way to do so would depend on the detaile... See more...
Splunk is not Excel But seriously. For Splunk every result row is... well, a separate row. Depending on the actual use case you could cheat a bit but the way to do so would depend on the detailed desired outcome. You could do something like <your_initial_search> | stats values(_raw) as "Event Details" by UID (and maybe do some magic with custom CSS in dashboard to "un-align" the table a bit). But that will give you just raw events. If you want to have separate fields from those "content" events... that's gonna get tricky and ugly (and un-splunky because the result will not have any internal logical consistency and will be only for presentation purposes). An example using my windows events index: index=winevents This is just the base search - nothing to write home about | sort EventID That should also be pretty obvious - we want the events grouped by EventID field. You can add subsequent sort field(s) if you want them sorted within those groups. | streamstats window=1 current=f last(EventID) as previousID Now the magic starts. We're copying the EvenID value from previous event to the current one. The previous one is called previousID. | eval splittable=if(NOT EventID=previousID,mvappend("1","0"),0) If the current EventID is the same as previous one (which we carried over in last step) it means that it's not the first result with given EventID. If those values are different (or - in case of the very first result row, the previousID is empty; that's why the condition is in the form of NOT a=b instead of a!=b), this is a first row of results for given EventID. Depending on which case it is, we create a temporary field with either a single value (whether it's a zero, or anything else is not important; I just chose zero) or two values of which the second one must be the same as for the "not-first" row. We're doing this because Splunk cannot just arbitrarily add rows. So we're doing the trick with multiple values in one result (so called multivalue field) so we can split that result into two separate ones. And this we do by calling: | mvexpand splittable Now the first row for each unique EventID, which we marked with two values in the field called "splittable" got split into two separate rows with one value each. The row which had just one value was left unchanged. What is also important is that the order of the split results remains the same as the order of the values in the field on which we're calling mvexpand. So now all that's left is to find the "header" row and clean all "non-header" values. And clean the "header" field (in our case the EventID field) for all "non-header" rows. | foreach *     [ eval <<FIELD>>=case(splittable=1 AND "<<FIELD>>"="EventID",EventID,splittable=1,"",splittable=0 AND "<<FIELD>>"="EventID","",1=1,<<FIELD>>) ] We may now remove the temporary fields which we don't need anymore (this step is optional if we're limiting displayed fields to a strictly defined set; if we just list all fields, we might want to do this so we don't drag temporary fields along) | fields - splittable previousID And now we can present the results as table with either | table EventID host _time field1 field2 and so on or simply | table EventID *   OK. So this exercise was fun but I wouldn't do that this way. After doing all this you're getting a set of results where you have no relationship between the EventID field from one result and the actual "contents" of the events in other results - you can't aggregate the data, (re)sort them or do anything else, maybe except some general statistics. This kind of result is unusable. As I said at the beginning - Splunk is not Excel and you can't "merge fields". The only way this could work would be if someone wrote a custom visualization which would do some JS magic comparing values from neighboring rows and fiddling with CSS but so far I don't think anyone did such thing.
| eventstats count as total by uid | where total > 4
I added sort by uid, and it did. It still shows the uid just as part of the big event record. What would be nice would be if I could pull it out to the side, so the recipient of the report could quic... See more...
I added sort by uid, and it did. It still shows the uid just as part of the big event record. What would be nice would be if I could pull it out to the side, so the recipient of the report could quickly see that uid 12345 had 5 events, and uid 67890 had 9 events, rather than just the detail event records. In other words (mocked up output): uid 12345       total number of events: 5    event detail1    event detail2    event detail3     etc. uid 67890       total number of events: 9     event detail1     event detail2     event detail3     etc.   Also, in case you couldn't tell, I am a beginner at Splunk. Thank you for your help.    
Hi @TheJagoff  How about this?  |makeresults | eval _raw="FAILED to copy checksum for: /logs/archives/archived-logs/server01.log.gz Host key verification failed. lost connection" | append [|makeres... See more...
Hi @TheJagoff  How about this?  |makeresults | eval _raw="FAILED to copy checksum for: /logs/archives/archived-logs/server01.log.gz Host key verification failed. lost connection" | append [|makeresults | eval _raw="FAILED to copy checksum for: /logs/archives/archived-logs/server02.log.gz You are attempting to access a system owned by XYZ Provide proper credentials for access Contact the system administrator for assistance ---This system is monitored--- Details as follows. scp: /logs/rsyslog/server02/: Not a directory"] | rex max_match=100 field=_raw "(?m)(?<message>[^\n\r]+)$" | eval last_line = if(typeof(mvfind(message,"Details as follows"))=="Number","", mvindex(message,-2))+" "+mvindex(message, -1) It joins the last 2 lines by a space for event 1 - might need tweaking to add the linebreak back in.  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
I guess this is because of the "by field" - Are you wanting to count by another field or just a total count? you can remove the by <fieldName> if you dont need that.  Did this answer help you? If ... See more...
I guess this is because of the "by field" - Are you wanting to count by another field or just a total count? you can remove the by <fieldName> if you dont need that.  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
I created a KV Store lookup using the "Splunk App for Lookup File Editing" app, however when I look at Settings>Lookups, the lookup definition doesn't show up.  In addition, when running | inputlook... See more...
I created a KV Store lookup using the "Splunk App for Lookup File Editing" app, however when I look at Settings>Lookups, the lookup definition doesn't show up.  In addition, when running | inputlookup <name> I get the error "The lookup table '<name>' requires a .csv or KV store lookup definition"   What do I miss? 
What distinguishes the first event from the second? Assuming it is a line with "lost connection", you could try something like this | makeresults | fields - _time | eval _raw="FAILED to copy checksu... See more...
What distinguishes the first event from the second? Assuming it is a line with "lost connection", you could try something like this | makeresults | fields - _time | eval _raw="FAILED to copy checksum for: /logs/archives/archived-logs/server01.log.gz Host key verification failed. lost connection" | append [| makeresults | fields - _time | eval _raw="FAILED to copy checksum for: /logs/archives/archived-logs/server02.log.gz You are attempting to access a system owned by XYZ Provide proper credentials for access Contact the system administrator for assistance ---This system is monitored--- Details as follows. scp: /logs/rsyslog/server02/: Not a directory"] | rex "(?m)FAILED to copy checksum for:[^\n]+\n([^\n]+\n)*(?!lost connection)(?<line>[^\n]+(\nlost connection|$))"
Very helpful, thank you. I will now play with eventstats, to try to refine the results. Using stats I had two rows, one for each where the count > X. Now with eventstats I get individual rows for eac... See more...
Very helpful, thank you. I will now play with eventstats, to try to refine the results. Using stats I had two rows, one for each where the count > X. Now with eventstats I get individual rows for each of the events that made up the two stats rows. So, for X=4, I had 2 rows with counts of 9 and 5, respectively. Now I'm seeing 14 events returned. Definitely closer to what I'm looking for.
@livehybrid  - I need the last 2 lines of the first event, and the last line of the second event. I honestly don't know if this is even possible. The events start with "FAILED to copy checksum for: ... See more...
@livehybrid  - I need the last 2 lines of the first event, and the last line of the second event. I honestly don't know if this is even possible. The events start with "FAILED to copy checksum for: " I will work with what you have sent and see what I get for results. Thank you.
I can not find anything in the outputs.conf that will allow you to control the HTTP version sourced at the UF itself.  Splunk documentation implies a LB can/should be used and can control HTTP versio... See more...
I can not find anything in the outputs.conf that will allow you to control the HTTP version sourced at the UF itself.  Splunk documentation implies a LB can/should be used and can control HTTP version.  Their example is NGINX but there are others out there which may or may not support in the same fashion. https://docs.splunk.com/Documentation/Forwarder/9.4.0/Forwarder/Configureforwardingwithoutputs.conf#Send_data_over_HTTP_using_a_load_balancer  
Hi @spm807  Once you have used "stats" you will have a statistics table with your summarised data output. At this point you are not able to view the original events.  Depending on your usecase you ... See more...
Hi @spm807  Once you have used "stats" you will have a statistics table with your summarised data output. At this point you are not able to view the original events.  Depending on your usecase you may find that "eventstats" is more useful? | eventstats count as total_count by some_field This will create the count (total_count in this example) whilst still retaining the original events.  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
How do I show details of individual records in a count total? I have a query that counts events, and then returns the total count when it's above a specified threshold. How do I display the individua... See more...
How do I show details of individual records in a count total? I have a query that counts events, and then returns the total count when it's above a specified threshold. How do I display the individual events that constitute that count total? But only for those totals where the count exceeds the threshold?
Hi @TheJagoff  Im struggling a little to work out the boundaries between the events but I think I might have it now...Just to check - is it the last line in each event that you want to extract? If s... See more...
Hi @TheJagoff  Im struggling a little to work out the boundaries between the events but I think I might have it now...Just to check - is it the last line in each event that you want to extract? If so the following might work well: | rex max_match=100 field=_raw "(?m)(?<message>[^\n\r]+)$" | eval last_line = mvindex(message, -1) Incase its useful for future responses, below is the full example with some makeresults to emulate your events. |makeresults | eval _raw="FAILED to copy checksum for: /logs/archives/archived-logs/server01.log.gz Host key verification failed. lost connection" | append [|makeresults | eval _raw="FAILED to copy checksum for: /logs/archives/archived-logs/server02.log.gz You are attempting to access a system owned by XYZ Provide proper credentials for access Contact the system administrator for assistance ---This system is monitored--- Details as follows. scp: /logs/rsyslog/server02/: Not a directory"] | rex max_match=100 field=_raw "(?m)(?<message>[^\n\r]+)$" | eval last_line = mvindex(message, -1)  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
Hi There isnt quite enough info in the post to work out exactly what you need - however the following should get you started. Note: I wouldnt recommend looking back 24 hours every time, what is the ... See more...
Hi There isnt quite enough info in the post to work out exactly what you need - however the following should get you started. Note: I wouldnt recommend looking back 24 hours every time, what is the reason for this? I would recommend just looking back 60 minutes, you could use earliest=-70m latest=-10m to make sure you get data which is up to 10 minutes late arriving. Run a search which returns the events you want to be alerted on: index=your_index action=update subcategory=WEB_DLP_POLICY earliest=-1d@d latest=now Click Save As-> Alert. Name the alert and change the settings to make it run hourly.  You'll need to make sure you apply some throttling because otherwise you may get the alerts duplicated every time it runs. Then scroll down and setup your chosen Alert action - presumably Email? Configure this according to your requirements and then save. Some useful docs: https://docs.splunk.com/Documentation/Splunk/9.4.1/Alert/Definescheduledalerts Did this answer help you? If so, please consider: Adding kudos to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
I have multiline events where it is required to capture the error messages. The events are separated by "FAILED". I need to capture "Host key verification failed" from the first event, "scp: /logs/... See more...
I have multiline events where it is required to capture the error messages. The events are separated by "FAILED". I need to capture "Host key verification failed" from the first event, "scp: /logs/rsyslog/server02/: Not a directory" from the second event. The events: FAILED to copy checksum for: /logs/archives/archived-logs/server01.log.gz Host key verification failed. lost connection FAILED to copy checksum for: /logs/archives/archived-logs/server02.log.gz You are attempting to access a system owned by XYZ Provide proper credentials for access Contact the system administrator for assistance ---This system is monitored--- Details as follows. scp: /logs/rsyslog/server02/: Not a directory   I can capture the first message with: FAILED.+\:\s(?<LogFile>.+)(\n)(?<Message>.+(\n).+) I don't know how to skip to capture the last line of the second event for the Message field. Any help is most appreciated. Thank you  
What have you tried so far?  Where did you get stuck? Why check the whole day every hour?  If nothing was found in the 00:00-01:00 period at the 01:00 run then nothing will be found in the same peri... See more...
What have you tried so far?  Where did you get stuck? Why check the whole day every hour?  If nothing was found in the 00:00-01:00 period at the 01:00 run then nothing will be found in the same period at the 02:00 run.  Searching the same data repeatedly is a waste of resources.
Would like to configure an alert that will trigger based on the action and subcategory below.  Would like this to run hourly to check if there are any hits daily. action=update subcategory=WEB_DLP_P... See more...
Would like to configure an alert that will trigger based on the action and subcategory below.  Would like this to run hourly to check if there are any hits daily. action=update subcategory=WEB_DLP_POLICY
Hi @bpenny  If you're looking to do it as an automatic lookup then you should be able to use the following, configured from Settings -> Lookups -> Automatic Lookups. Or as a props.conf: [yourS... See more...
Hi @bpenny  If you're looking to do it as an automatic lookup then you should be able to use the following, configured from Settings -> Lookups -> Automatic Lookups. Or as a props.conf: [yourSourceType] LOOKUP-lookup1 = yourLookupName type AS "msg.message_set{}.type" OUTPUTNEW typeDescription AS typeDescription  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing