OK. Fully honestly, I don't see the point in doing so. You're adding some synthetic zeros and then count something which will easily be divisible by 4 (because when you're preparing your bitstring he ones must be separated by multiplies of 4). You're doing some strange bit swapping (and it's completely inconsistent - your original example had something that resembled a 16-bits little-endian integer which you "converted" to 32-bit value and from my single bit 0x63 value you also built a 32-bit value but obviously in a different way). So, to be absolutely frank, it doesn't make much sense. A completely separate thing is why would you want to do it in splunk? (actually the easiest solution could be to implement an external lookup using a python script). ... View more

1. Don't use the _json sourcetype. If needed, create a new one copying settings from the _json. 2. Don't use indexed extractions unless you're absolutely sure what you're doing. 3. Don't edit the _json sourcetype - it's a builtin sourcetype which shouldn't be used explicitly anyway (see p.1) 4. The count(field) aggregation counts single values so if you have multivalued fields it's normal to have count(field) higher than a general event count. In your case you probably (as @gcusello already pointed out) have multiple occurrences of a "timestamp" field within a single event so it gets parsed as multivalued field and gets counted accordingly. ... View more

Unfortunately, third-party addons and their manuals are often... how to say it gently... not written in the best way possible. They are written by people who might be proficient with their respective solutions but not necessarily knowledgeable in Splunk. The advised way to get syslog data to Splunk is still using an external syslog daemon which will either write the data to files from which you'll pick up the events with UF and monitor input or which will send the data to Splunk's HEC input. For a small-scale test environment sending directly to Splunk might be relatively OK (when you don't mind the cons of such setup) but you need to create your udp or tcp inputs on high ports (over 1024) when not running Splunk as root. ... View more

Change RF/SF to 1 and the CM will not complain about missing nodes. ... View more

If you simply want to find out when a host started sending data to index you simply need to find min(_time). | tstats min(_time) where index=something earliest=1 by host Two caveats 1) It's based on the _time field so if you've ingested a backlog of 3 years worth of data right after deploying your forwarder, your results will probably not be true. I don't remember if you can use _indextime in tstats. You have to check. 2) It will of course only show data from the buckets which haven't yet rolled to frozen so for old data it will not be true ... View more

There are more or less three ways of going about it. 1. Freezing the data to external storage instead of removing it - the downside is that you have to thaw the data if you ever want to use it again. 2. Simply stop your server and copy out the indexed data from the buckets - it uses much more space but you can copy those buckets back into index directory and you're ready to go (unless you forget about retention periods and your data immediately rolls to frozen ;-)) 3. Bend over backwards and run a bunch of searches exporting your data to some csv or json. The upside is that you can use such export with other tools (probably after some processing) but the downside is that you won't be able to use it again with Splunk without additional magic and reingesting it into index. ... View more

Adding to @dural_yyz 's answer - your question seems to not be Splunk related but rather connected with your source system which might or might not be able to produce required logs. If you're not getting logs into Splunk, assuming that the intermediate sc4s is working in general because it sends other logs, there are two posibilities - either your sc4s is misconfigured and doesn't send the data properly (but to troubleshoot that you'd need to be absolutely sure that sc4s is getting relevant events from the source; did you verify it?) or your source is not sending the desired data (and this is something you need to resolve on the source side). ... View more

You could attach your props to some wildcarded host or source stanza but that's something I'd be very careful about. It's a very non-obvious configuration and can be a huge pain to debug issues. ... View more

The "query" (or in Splunk terminology - search) you're looking for will depend on what data you have indexed in your Splunk. ... View more

Hi @hazem , good for you, see next time! Ciao and happy splunking Giuseppe P.S.: Karma Points are appreciated by all the contributors 😉 ... View more

Yep you are right, I wasn't thinking about that. We can still use the following, | streamstats count as Rank | delta Score as Diff | eval Rank=if(Diff=0,null,Rank) | filldown | fields - Diff ... View more

Let's add some additional stuff to the mix. 1. Raw number of events is one thing, but their size also matters. It's a different thing to send 1000 of short syslog-received messages and completely another thing to send 1000 several kilobytes long stack dumps from java app. 2. Technically at some point you will hit some limit (after all server memory doesn't grow on trees ;-). But probably sending tenths of iso images within a single batch request isn't what you're aiming at 🙂 3. And finally, even if you're not using acks, will your source be able to resend the events batch from a given point should any error happen in the middle of the batch and only some events were accepted? ... View more

That's understandable. Your files consist mostly of a relatively constant part repeated across all files (the header and some relatively constant fields) so Splunk will be guessing that it's all the same file. If the filenames are unique and the files are not rotated in any way, you can use crcSalt=<SOURCE> (That's actually one of the rare cases it can actually make sense). Otherwise, raise initCrcLength so that it catches variable parts of the event. As a side note, it seems that the event is very verbose and could use some serious editing on ingest to save on license (you don't need majority of the raw data). Additional questin is whether there should be any event breaking done within a single fioe. ... View more

Actually ingesting via an S3 bucket is a fairly unusual scenario. Start easy - by deploying a UF on a windows box and reading its eventlog channels. Then try ingesting data with file monitor inputs. Then you can try installing some apps with modular inputs and configuring them. And actually, adding data is not really much of a cybersecurity task. It's more of an admin chore. ... View more

We can only see that the server is throwing a 500 error. We can't tell why. There should be something more in the logs. Check out _internal to see what's going on. ... View more

Sorry, mate, but the level of completness of your description is comparable to "ok, I replaced the flat tyre but now I cannot put the car in gear - can it be a problem with a battery?". We nave no idea about what your setup looks like, what hosts you have, what configs. How can we know what's wrong? ... View more

Well... the typical approach is to get a piece of paper (or some excel sheet) and list all (kinds of) sources you're gonna be getting the events from. And work with that - think who should have access to that data, how much of that data you're gonna need, for how long it will have to be stored, what are use cases you'll be implementing using this data (because that also can affect the data distribution). It's a bit complicated (and that's why Splunk Certified Architect certification is something you get only after you've already certified for Admin and while the course itself might be taken before that there's not much point in doing so) and it's hard to say in a short post about all possible caveats regarding proper indexes architecture. But bear in mind that indexes in Splunk do not define the data contained within them in any way (at least from the technical point of view). They are just... "sacks" for data thrown in. And you can - if needed - have several different "kinds" of data within one index and usually (unless you have some strange border case) it doesn't matter and doesn't give you a significant performance penalty. You simply choose some of that data when searching by specifying metadata fields (like host, source, sourcetype) as your search terms. One exception here (which I'll not dig into since we're apparently not talking about it at the moment - there are two types of indexes - event indexes and metrics indexes). ... View more

Splunk doesn't look "backwards" so you have to think backwards 🙂 So as Splunk by default returns events in reverse chronological order, you have to | reverse them to get them in straight chronological order. 2. Assuming that you already have the REQ field extracted, keep track of its values for a 7-minute long window | streamstats time_window=7m values(REQ) AS reqvals 3. Now you can find those events matching your searchnstring and not having the value of REQ copied over from earlier events | search "Error occurred during message exchange" AND NOT reqvals="INI" Two caveats 1. The search might be slow. Depending on your actual data you might make it faster by searching only for "Error occurred during message exchange" OR REQ 2. Remember that a!=b is not the same as NOT a=b. Especially when dealing with multivalued fields. ... View more

Ok, you're mixing several things here. One thing is syslog - UFs don't send syslog. They can receive syslog (which is not advisable anyway to do directly on UF but that's a other story). They send data over so-called s2s protocol which might be embedded in http requests. UFs have their own load-balancing mechanism and you should be using that. Network-level load balancing won't work properly. If you expect your indexers set to change frequently you might use indexer discovery. This is all you should discuss with your Splunk environment architect. ... View more

thanks sir 🙂  I was thinking something complex, but you made it very simple. ... View more

Hi @tomjb94 , good for you, see next time! let us know if we can help you more, or, please, accept one answer for the other people of Community. Ciao and happy splunking Giuseppe P.S.: Karma Points are appreciated by all the contributors 😉 ... View more

Hi @Ram2 ...May i know what happens when you try this props: SHOULD_LINEMERGE=false LINE_BREAKER=([\r\n]+) TIME_PREFIX=^   ... View more

Hi PickleRick i was going from 9.0.1 to 9.3.0. cheers,  Dabbsy ... View more

Of course you're using inputs.conf. Without it you'd have no inputs. Question is what inputs you get your data from. Is it a simple tcp:// or udp:// input and you're receiving data directly on your indexer (which you shouldn't)? Is it an intermediate syslog daemon writing to files which are read by UF? Is it something else? ... View more

I agree with @PickleRick that using lookup might be more performant if the lookup file is not very large and there are not many matches.  If the lookup is very large, you can eliminate one subsearch because there is only one lookup. index=network_traffic NOT [inputlookup devices.csv | stats values(ip_address) AS src_ip | eval dest_ip = src_ip]   ... View more