And of course, the very next link I hadn't yet followed yet, was a answer by the inimitable Martin, https://community.splunk.com/t5/Archive/How-do-you-interpret-string-variable-as-SPL-in-Map-function/m-p/385353 which if you can create a macro do magic on your string, then this can be done. It's totally a hack, but at least it's something. ... View more

I was looking for an answer to the same sort of question, if one exists, and as I read this exchange I wanted to comment.  Partly as a "better but still very unsatisfactory answer" for folks, and maybe also for a bit of "let's see if anyone else comes up with a solution". I think the ask is really for - Is there a way to have a code path that either is or is not *executed* depending on some condition; a variable being set or not, or being of a particular value or whatever? In every example thus far, the code path executes regardless of the status of the variable.  You can test this yourself with a search/expression there that fails and fails *hard*.  Here's an example. On Splunk <any edition but free> this returns either your srchJobsQuota value, or the word "skipped" depending on if you flip the true to false.  To be clear, this works fine: | makeresults | eval results = if(false(), [| rest /services/authorization/roles/admin splunk_server=local | return $srchJobsQuota], "") But if I change my test box from my dev license to the Splunk Free license, since Splunk Free doesn't support that rest call *at all* (LOL - "Payment Required"), then no matter which way false/true is flipped, this fails. In both cases in Splunk Free, true and false, you get this same error: Error in 'eval' command: The expression is malformed. An unexpected character is reached at ', "skipped")'. [subsearch]: Unexpected status for to fetch REST endpoint uri=https://127.0.0.1:8089/services/authorization/roles/admin?count=0 from server=https://127.0.0.1:8089 - Payment Required So it's executing that code without regard to whether it's in the active path or not.  Because it's a subsearch, they run first. I thought maybe I could get around this using map, but all attempts so far that I've tried have failed because I can't find a way to run an arbitrary string (which can easily be created from an if like above) from map.  It looks almost like you should be able to, but either it just won't work or I haven't found the magic combination.  And I've tried quite a bit... I'll think on it some more, maybe there will be more suggestions from the peanut gallery too.  🙂 Happy Splunking, Rich   ... View more

Does this involve Splunk at any point? I'm not sure this is the best forum to be asking this question. This is the Splunk community answers platform, not the biztrak help forums nor the "basic networking" help forums. We really do like to help, and sometimes we'll reach out a little and help with some basic networking problems, but mostly only when it involves Splunk.  It's just not the best use of our time when the right answer is to use your favorite search engine and type in "ssl vpn configuration" or "biztrak database not running". But if you have a Splunk question later, please do ask! ... View more

I bet it's DNS, because it's always DNS! (For future reference, your question is nearly impossible to answer as it's been asked.  Please review this: https://docs.splunk.com/Documentation/Splunkbase/splunkbase/Answers/Questions and maybe post a new question after you've thought through what we'll need to help you answer it? )   ... View more

Yes, streamstats. earliest=-2h*h latest=now() index=_internal blocked=true name IN (<that list>) | bin span=1h _time | stats count() AS MyEventCount by _time | streamstats window=2 earliest(MyEventCount) AS previous_count, latest(MyEventCount) AS latest_count | eval difference = previous_count - latest_count I did change it from a `| timechart ...` to a `| bin _time | stats count...` but both accomplish the same goal, so you can really use either. Streamstats in this case chops up that data into 2-event chunks, taking the earlier myEventCount as the previous count, the later one as latest_count.  The last we just do an eval to create a new field difference with the ... difference.  🙂 Also 100% guarantee I did my math backwards there.  I get 'em wrong so often, I no longer try, I just warn folks to switch that subtraction around if necessary.   ... View more

You should probably ask a new question, since this answer isn't really part of the original. But why not build your own? | makeresults | eval lat1=45, lon1=-90, lat2=39, lon2=-84 | eval earth_diameter_km = 6371 | eval lat1_rad = lat1 * (3.14159 / 180), lat2_rad = lat2 * (3.14159 / 180), lon1_rad = lon1 * (3.14159 / 180), lon2_rad = lon2 * (3.14159 / 180) | eval diff_rad_lat = lat1_rad - lat2_rad | eval diff_rad_lon = lon1_rad - lon2_rad | eval calc1 = pow(sin(diff_rad_lat/2),2) + cos(lat1_rad) * cos(lat2_rad) * pow(sin(diff_rad_lon),2) | eval calc2 = atan2(sqrt(calc1), sqrt(1-calc1)) | eval distance_km = earth_diameter_km * calc2 That's a run-anywhere that roughly calculates the distance between my house and my parent's house.  It's based loosely on this: https://stackoverflow.com/questions/27928/calculate-distance-between-two-latitude-longitude-points-haversine-formula But understanding what I wrote there is an exercise for the reader.  🙂 ... View more

Don't union or join.  Both of those commands usually have better options available in Splunk so should only be used as a last resort. A lot of how you'd want to pull them in together depends on a) Do they have a field in common to group them by? b) How much data are we talking about, and how large *could* it get? c) Does either of the RDBMS's behind the scenes have access to the other one?  (You have two connections, but both pulling from "fo" so maybe they're part of the same actual DB?) If C is true, then by far the best solution is to join these in the RDBMS, which you can do in Splunk. select date.description member.member, member.gate, member.port from fo.member member INNER JOIN fo.date date on date.<date-side-field-to-join-on> = member.<member-side-field-to-join-on> NOTE that's ONLY the SQL part of things.  You have to work with your DBAs to figure out what that query looks like.  But once you have it, build a ... I do wonder why you have two separate connections to what appears to be possibly just two tables in a single DB? In any case, you would then either build a view in the DB and query that instead, or use the search you came up with as the query inside dbxquery. If you know they're small and will stay so (A&B above) , the general pattern is |dbxquery query="select member, gate, port from fo.member connection=fo_member" | append [ |dbxquery query="select description from fo.date connection=datelog" ] | stats values(member) values(gate)... BY <field-they-have-in-common> But it's limited to 50,000 rows, and will just silently drop everything after 60 seconds or the 50,000 rows is reached.  Please reread that - I mean *silently* drop anything beyond that. At second 60, it's autofinalized and returns whatever it had, even if it's not complete. If it hits result 50,000, it stops right there and returns the 50,000 it had, ignoring any additional rows it could have retrieved.  In both cases, it won't even tell you that it's done so, just stops pulling back results. But hey, if it's small and quick, it's fine.  🙂 Happy Splunking -Rich ... View more

Probably the easiest way is to 1) Run a search (doesn't matter what search) 2) Open the Search Job Inspector https://docs.splunk.com/Documentation/Splunk/8.1.0/Search/ViewsearchjobpropertieswiththeJobInspector 3) Expand the section for "Search job properties" 4) Scroll down to the section "searchProviders" and that's your list of indexers it talked to for this search.   Happy Splunking, Rich   ... View more

I think you are almost there. You don't have two columns in your CSV file, so ... you can either add one, or you can just OUTPUT the original field as "found".  index=X | lookup bad_user_agent user_agent OUTPUT user_agent AS found | search found=*   Do note that there are definitely other ways to do this, too.  Most have more side effects than this way does though, or is more fiddly and finicky or is less scalable. ... View more

For the first thing - the TGZ installation method is literally the first one listed in the installation documents: https://docs.splunk.com/Documentation/Splunk/8.1.0/Installation/InstallonLinux For the second thing (networking error) In any case, your error is cause by a misconfiguration of one of the below: a) Your local or network firewall b) Your system's network configuration c) Your network's configuration "No route to host" is a standard error meaning that your system has no "route" through which it can send networking packets to your Splunk server.  If it can't send networking packets, it can't send Splunk data.  This has nothing to do with Splunk, as you can prove by opening up a terminal on the UF and typing 'ping X.X.X.X' substituting your splunk server's IP there.  That should give you an error very similar to the one you see in the Splunk logs. To resolve, open up Google, or Bing, or DuckDuckGo or any other search engine and type in "no route to host".  When you get results, read a few, think about how the things they describe might be the case, and investigate.  It might take looking at a few sites, following a few commands using netstat and traceroute to figure out where the problem is, and revising your searches and approach to fixing it. To open a ticket, go to the Support Portal for Splunk and click create ticket, or whatever that button is.  Since you are talking about a UF, I assume you have a Splunk server with probably an actual license installed on it, so you should have support.  If that's not the case, well,  I hope a) you aren't using this for production purposes and b) maybe you should pay for support... ? If you tried to open on on "no route to host" they're probably going to tell you to "fix your networking" 🙂   But the failure to be able to install might get some traction with them. -Rich   ... View more

A short term solution - and the one I  use anyway - is to use the .tgz installer.  It takes the additional steps of a) creating a user, b) chown -R on the /opt/splunk directory to change it's ownership to that new user, and c) setting up boot-enable manually.  But it's all easy enough.  I do not believe it's recommended to mix and match a .deb install with a .tgz upgrade or vice versa, so you may want to use this with a little bit of caution to make sure you know *this* forwarder has the .tgz on, but otherwise it should work OK. Also why not open a support ticket for this?  Sounds like an appropriate step. ... View more

Well, I have some ideas but you never mention *which* field you want the length of (and to limit to ones that are 7 characters).  Anyway, no worries, it's just a single eval to get the length, and you can search again after you make that eval if you want to filter it down to just ones of length 7. The eval to create a new field called 'fieldLength', which contains the number of characters in the field "fieldX" | eval fieldLength=len(<fieldX>) I recommend adding that and checking that it does what you want (which probably means you want to drop the "chart" command off temporarily). If you want to filter it to the ones that are length 7, easily enough you'd just do like the above only add a line to search. | eval fieldLength=len(<fieldX>) | search fieldLength = 7 You'll have to modify it to fit your fields, but that's about all it should take. Happy Splunking! -Rich ... View more

Ah, those sums. I'm not quite sure if you then answered your own question? In any case, leave off the `| stats ... ` and everything after it, and you'll see that the field 'sum_for_direct_attive` and that other field are both either 0 or 1 depending on the eval we did earlier.  Which that eval is really just a replacement for the previous examples inline evals in the stats command itself. So, since we've created those fields before the stats command, we can just sum up the 1's and 0's of those events appropriately. inline evals in stats can be great, but sometimes they just make the stats command really complex looking, so sometimes it's better to just split those out into their own little bits of logic so they're easier to understand and work on. And in case you didn't figure it out, "-3mon@mon" would go back to August 1st.  You can mix and match those too.  "-90d" goes back 90 days from this time right now, but "-90d@d" goes back to the midnight that happened just before day 90 ago. "-3mon@d" goes back three months, then backs up until midnight of that day to include the entire day from 3 months ago, instead of only the partial day.   ... View more

(Also never mind I'm not moving messages, that function does not do what I thought it does, and looks pretty scary.  🙂 ) ... View more

Aww, thank you.  I'm glad you found it useful. Your changes look perfect to me - and I had to laugh, I usually write when I do division to come up with percentages that I *always* do the math backwards and that you'll probably have to fix it.  Glad you noticed without me telling you to. For the sum in the stats - Well, the calculations were going to be harder to do an actual count, or maybe not harder, but would look less clear?  Counting things is really just summing up "1" each time you count an item, right?  So I've gotten used to, whenever I'm dealing with a *conditional* count to use the sum of 1 when it matches, 0 when it doesn't.  It just seems easier to understand. (And obviously, for simple cases where you just want a count, you just `stats count by _time` or whatever, plain old "count" works fantastically well in that case.) Excellent work on fixing me having forgotten your 45 day limit.  That's probably why my numbers didn't match yours.  🙂 For your last question on dates and old data, I guess the question here really is, how is this data being ingested?  In my mind, the rightest answer to this problem is to re-ingest the data (I assume it's not huge and it's in a text file somewhere?) and when you set up that input make sure it is reading the dates correctly, and using the right date field as the date to use for _time.  You can see I hacked in a way to make it work, assigning _time to one of the interpreted date fields using eval, but it would be better to just have that step done on the input itself. (https://docs.splunk.com/Documentation/Splunk/8.1.0/Data/Configuretimestamprecognition) If you did that, then your time picker would work.  You could just pick "last 3 months" from the time picker in the upper right, and it'd filter everything correctly. But, since we don't have that set up yet, I did a thing: | eval _time = dataout | eval oldest = relative_time(now(), "-3mon@mon"), newest = relative_time(now(), "@mon") | where _time > oldest AND _time < newest | bin span=1mon _time I didn't include the whole mess, just included the two lines you can add (and a line before and after it so you can see where it fits - after we eval a _time, and before we do the bin on _time.). What I do there is create two fields using a function in splunk called "relative_time".  Here's how relative time works and in there is other links to various other things that may help understanding it. https://docs.splunk.com/Documentation/SCS/current/Search/Specifyrelativetime The first field is the oldest records we want to keep, so I set it to "-3 months ago (e.g. "before now()"), snapped to the beginning of that month" with the setting "-3mon@mon".   So run here in the middle of November 18th, it'll go back three months to August 18th then back to the beginning of that month, August 1.  I also snapped the newest records we want to the beginning of this month, Nov 1. You actually probably don't want any snapping, now that I've said all that. I think you probably want those to read "-3mon" and not even create the newest value at all.  Then obviously if that's the case your where gets simpler.  So if you don't want the snapping, try: | eval _time = dataout | eval oldest = relative_time(now(), "-3mon") | where _time > oldest | bin span=1mon _time   In any case, I think you know where you'll have to look (the input and whatnot) for fixing this "generically" so the time picker works, and that even without this, you should have you a reasonable workaround.    Let me know if that all works!   ... View more

All right, two things - one super minor bookkeeping thing that I was doing to, another that is actually pertinent. Minor bookkeeping thing - I keep clicking the wrong "reply" button so the threading is breaking. Either that or the new platforms threading is still bad, not sure.  Anyway, I am a moderator so I am going to start trying to shuffle things around to thread this all right. The real thing - Let's check the generated LISPY for each thing. For a search that works, and one that doesn't, run the search.  Then click the job dropdown near the top of the data/output pane.  Select "Inspect job" and it'll open a new window.  In there, click the link to open the search.log (it's either a link right near the top, or all the way at the bottom). Search in there for "base lispy" and when you find it, copy that out and paste it back into an answer here. It may be useful to search for ERROR and see if anything matches that and include it too.   Then repeat with a search that doesn't behave properly. I'm not yet sure what this will tell us, but maybe it'll be useful.  It often is. -Rich   ... View more

Also maybe just try index="nameofourindex" test tag="snooze" | stats count by tag Possibly we'll see something useful in that.   ... View more

I am puzzled then. Back to basics, then, let's see what thing we're missing. index="nameofourindex" test Returns 12 events? index="nameofourindex" test tag="snooze" Returns some smaller number of events, let's say N. index="nameofourindex" test NOT tag="snooze" Should therefore return 12-N=M events, so if N is 4 from the above, this would be 8.   Is that or is that not the case?  If your search is a little more complex, please post it here in case there's some nuance with parenthesis or something we need to investigate. Don't worry, we'll get this.  Just might take a bit of slow going for a bit to make sure we're not overlooking that one thing... 🙂 -Rich ... View more

Oh I wonder if they're not being parsed out properly as fields, so it's really doing string comparisons - So `NOT tag=X` is actually looking to do a `NOT <string "tag=X">` instead of `NOT <field tag>=<value X>`. I'm trying to wrap my brain around that, if it's possible, and if it's possible is that the problem here? OK, let's leave that aside for just a second, I'll get back to it. So here's how it should work - probably all stuff you've figured out, but feels useful to just get it out here for others.  If you run this search (all by itself), you'll see I mock up some data, LOG1-LOG5 events.  Then I conditionally create a tag field, for LOG2 and LOG4, it's set to "snooze".    For grins and giggles, I also set LOG3's tag to "otherTag" just to illustrate all the possibilities. | makeresults | eval Type="LOG1 LOG2 LOG3 LOG4 LOG5" | makemv Type | mvexpand Type | eval tag = if(Type=="LOG2" OR Type=="LOG4", "snooze", null()) | eval tag = if(Type=="LOG3", "otherTag", tag) Now, to the end of that if you append | search NOT tag=snooze It will return three rows; the two (LOG1, LOG5) where tag isn't defined at all, and LOG3 where tag is defined but it's not "snooze" Alternatively, if you change that last trailing `| search ...` to be | search tag!=snooze You'll get back one row for LOG3, because it's the only row that has tag defined, and yet where tag is not "snooze". So, to come full circle, here's what I think is happening. Your "field" isn't a field, and this needs to be fixed.  You don't have a field named "tag" with a value or or not of "snooze", you instead have a string that has an equals sign in it. "tag=snooze". The way to tell is, if it's a real field, run your search in Verbose mode (upper right, by the time picker - Fast, Smart and Verbose dropdown), and then on the left hand side check the fields - if you don't have one called "tags" which when you click on it it gives you values like 'snooze' and whatever, then it's not a "field" yet and is just a string. Automatic field extraction should have taken care of this, and since it didn't I suspect you have a data onboarding issue.  Perhaps you need `KV_MODE = json` on the stanza in props.conf for that sourcetype?  You can see that information here: https://docs.splunk.com/Documentation/Splunk/8.1.0/Knowledge/Automatickey-valuefieldextractionsatsearch-time There's ways to fix this after the fact (actually, KV_MODE = json fixes it after the fact too because it's search time), but maybe you can run ` | spath blah blah` on it, or perhaps in this one case we can eval it into a new field or rex it.  But either way, investigate why it's not a field and get that fixed, and I think everything else will just work.   Let us know what you find, Rich ... View more

Really!  huh. Can you provide an example of each category of event? Because no, except for the typo I probably made "sooze" vs. "snooze" I can't think of any reason this wouldn't work so I'm a little puzzled.  Feels like if it were an actual bug, it would affect nearly everyone and be well known, and since that's not the case I suspect we're just missing something simple here. Which means *sigh* we start with the basics of getting a few lines of sample data.  You can clean them up and obfuscate bits if you need to, and I don't think we need all fields, just a representative sample of them with the important fields there (e.g. "tag" lol).  Also the actual search you are using, if possible? ... View more

Do you have a few lines worth of the original data you can provide?  I'm having a hard time envisioning what it is we're doing.  It might also help if you could rephrase what it is you want. You've told us in the beginning how it is you expect to calculate what it is, but not ... not what it actually is.  Is it calculating the percentage of KPIs "complete" within 45 days of first being noted? Lastly, just on a whim, I notice you created a field called `Month` but then didn't use it to split your timechart by. I don't think this will help you, but try changing your last line to | timechart count(DATA_OUT) BY Month You probably already tried that one, and it'll be similar to the thing you already have written. But I figured, it is worth a shot just in case it's that tiny change makes it all better for you!   Anyway, sample data please? Thanks and happy Splunking, Rich ... View more

There's a common pattern to doing this.  Assuming we have no problems with lookups with special characters in them (I *think* that forward slashes won't bother anything? that semicolon may mess it up but I'm not sure... anyway, this is all testable, and able to be worked around if it causes problems!)... And, I hope you have at least one other field in that lookup?  Let's assume for a second you have a second field in there, you can use it like so: index=foo sourcetype=weblog extra_search_stuff_goes_here | lookup <yourlookupname> user_agent OUTPUT <fieldX> AS found | search found=* That would snag all your web logs, then run a lookup against them using your lookup, using the user_agent as the key, and would output the contents of that other field you had into a new field named "found".  Lastly, just search for where, after all that, the "found" field is there and set to something. If you don't have a second field in the CSV lookup yet, you can add a field to it and make it easy for yourself by calling it "found" and setting it to 1 everywhere in the CSV.  Then your search is a little simpler.  index=foo sourcetype=weblog extra_search_stuff_goes_here | lookup <yourlookupname> user_agent OUTPUT found | search found=* Right?  Because if found is already a field in the CSV, then ... we just output that. If you find that, say, forward slashes cause problems, you can remove them with rex from your data before doing the lookup, and in that case just remove them from the lookup too before resaving the csv file. in any case, hopefully this answers your question. If it doesn't, then by all means let me know what we've missed and we can try again. Also if this didn't answer your question, I can tell you in no uncertain terms that the solution will NOT involve shenanigans with `inputlookup` or `join`.  So don't fall for the bad answers out there trying to get you to use those. Happy Splunking,. Rich ... View more

Let's get started by me asking this question.  If you were to do this by hand, using those two records above, how would you do that?  If CLIENT_IP_ADDRESS is actually that word, I don't see how they even connect.  You can't do it by hand, and Splunk can't really do it either - there's no magic involved. If CLIENT_IP_ADDRESS is a stand in, and actually IS 192.168.10.10, then we have a chance. You'll probably need a better regex than this (regex101 is your friend!) because I'm making it up anchoring on something in the middle that might or might not be a good anchor, and I'm too lazy to build a regex from the beginning because I'm not actually sure that's a 100% accurate sample event.  But here it is. (search that returns the first set of events) OR (search that returns the second set of events) | rex "Info: \d+\.\d+\s+\d+\s+(?<client_ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})" | eval client_ip = SrcIP | stats list(DstIP) as Destination_IP_addresses BY client_ip There's two searches in the first line, connected together with an OR - you have to pile all the data into one place, then let Splunk sift through it and connect things together as required. The second line uses a regular expression to pull a new field called "client_ip" from what *might* be the IP address in there.  This will need fiddling with possibly, as mentioned above. The third relies on the fact that there is no "client_ip" in existence in any event with a field named SrcIP (an assumption, but seems legit given what I can see?), so you just create a new client_ip from SrcIP.  Between line 2 and line 3, we've now let both "sides" have a client_ip that matches each other, so we can group on them in a variety of ways. Line 4 uses stats, which probably should be called "group" in this case because really that's what we're doing, to list all the DstIP addresses by client_ip. For that last line, you can use values(), list(), or any other stats calculation, on any number of fields all at once.  Like "values(DstIP), sum(initiatorBytes), list(Protocol)..." Look up "splunk stats site:docs.splunk.com" and it should totally give you all sorts of good advice. The same mechanisms can be used to include other fields in the output, or to join things up in slightly different ways.  Hopefully this gets you your answer, but even if not I hope it does get you started on fine tuning your question for us, we're happy to take another pass at it! Happy Splunking, Rich ... View more