Hello @HathMH ! Welcome to Splunk and to the working world! Essentially what this error implies (and what the solution you mentioned was talking about) is that you've got data coming in, where you have perhaps not given Splunk enough information about how to handle the timestamp. It has chosen something... either something that looks like but is not the timestamp OR there is a level of granularity in milliseconds to the timestamp that is proving too much for the hardware/networking you are working with, without some kind of care and feeding. When data is brought into Splunk, and you don't give it instruction, it will do its very best to first figure out where a good timestamp is and it will use it. Then it looks for key=value pairs etc... and a number of other things in order to make sense of your data.  It works hard for this and some data will cause it to work unreasonably hard. i.e. the data looks one way for a while and then suddenly it changes so Splunk has to re-evaluate it's decisions.  you know your data best... so it is best to share that info. This is what the other solution is talking about: https://docs.splunk.com/Documentation/Splunk/9.0.0/Data/Configuretimestamprecognition Now... if you need this granularity that you're getting and all the time recognition is intentional, then you have to look at how the indexers are configured and how the index=mail is configured regarding how much it collects into each type of bucket before it rolls.   you should not START with complex bucket math. And it isn't recommended to do too much tinkering with the index configurations until you know what you're doing (meaning you are a certified admin and have had some experience at it).   Take a look at the sourcetypes in the index called mail index=mail |stats count by sourcetype you need to look for the props.conf that mentions that sourcetype and look for the timestamp configuration. If it's there... then you might need more help. If not... then identify the timestamp and apply that to the data on the way in. warning... don't do this on a production system. Best thing to do is to take a sample log, set up a test index and send data into it with your test config. Maybe you have a dev or test env... try it there. ... View more

Hi @Will_powr you haven't shown us what your data looks like so it is not as simple as showing you "how". So I will direct you to some background information and the "why" regarding usage of the Splunk Common Information Model CIM and how it works in Splunk (with Data Models) The What and the Why: https://docs.splunk.com/Documentation/CIM/5.0.1/User/Overview The How To: https://docs.splunk.com/Documentation/CIM/5.0.1/User/Howtousethesereferencetables It isn't necessary to normalize your data, but if you want your switch data to show up in a Splunk App that utilizes the CIM (maps fields from your data into the data models so that a search using the data model fields will work on your data automagically) you should look into it. You can also go down this fun looking rabbit hole: https://lantern.splunk.com/Splunk_Platform/Data_Application/Data_Types/Network_switch_data ... View more

yes. a lot of editing and "trust of links" comes from raising your Karma. Spend a bit of time here and keep answering things. The more accepts you get the higher your karma will go. I believe you need to hit 1000 before the editing capabilities open up. ... View more

Thanks for the explanation (now gone as I've removed my downvote). I was unaware that there was a link in the marketplace for Splunk Cloud I've edited your accepted answer with your addendum so that it will come up in full in searches (in google etc) up front. FWIW Certs are self signed across the board. You can replace with your own certs if you wish. ... View more

Each panel on a dashboard has a Splunk Search at its origin. The search you created for the server up time has fields that you can display in a table or chart, or you can pull specific values out of that search result and display it using tokens. So however you are going to display it... it must begin with some kind of search. If you would like the applications on each server to display dynamically (i.e. when someone drills down on each server) then you'll be taking the server name/address value and passing it via token to the second search so that you are listing the apps from a particular server. In order to do that there are several ways one might use that server value. The most straight forward would be using the search server=$tok_blah$ example. where the value of the token is used as a filter in your second search. So It might look like index=foo |search server=$tok_blah$|stats count by application_name Or you might say instead `index=foo|stats list(application_name) by $tok_blah$ You'll want to look here: doc on tokens and dashboards And here: which discusses contextual drilldowns in dashboards Hopefully this will help get you started in the right direction. ... View more

When you first get started with configuring data architecture for logging to the cloud, it can be confusing as to which is pointing to what. So first I'll suggest that you scan through this doc to be sure that you have everything aligned the way we expect it to be. Just on the off chance you've got something pointing in the wrong direction. (I don't think so from your description... but what your saying isn't making a whole lot of contextual sense, so let's just say - you will want to go over the configuration carefully) ClickHere To Read About Sending Data To Splunk Cloud From A Forwarder The link starts with the instructions for Windows. ("unknown folder" suggests to me you're on windows but all the instructions for windows, linux and macOS are there one after the other) The Data you are Asking the UF to read will be read and forwarded to the indexer in Splunk Cloud. However, if it is NOT making it to SplunkCloud for some reason... it isn't going to be physically dropped on the floor. i.e. nothing is going to make a folder without a name. Meaning... the UF won't just spit it back out. So something you are doing is making that folder... Now if that context and the doc doesn't help you. What we need to give you a hand is for you to show us the inputs instructions you've added and the outputs this will be found in the configuration files inputs.conf and outputs.conf which may have been deliberately created by you or created by using commands at the commandline (which will make directing you to them a bit more complex) If you need help finding them... you will need to provide a bit more context to how you are doing things and i'm sure we'll be able to get to the bottom of this for you. ... View more

I wonder if some of your terminology is keeping folks from being able to form a constructive answer... a lookup in Splunk is one of several formats, but they are all specific and structured. The delimiter is specific. https://docs.splunk.com/Documentation/Splunk/7.3.0/Knowledge/Aboutlookupsandfieldactions Depending on how large and dynamic this file is, you might be better off pre-processing it and then feeding it in as a lookup either to the KV store or as a csv. But you could also legitimately read that file (if it is very dynamic and perhaps very large) into an index (you can have as many indexes as you like) using whatever you like as your delimiter. When the destination is an index... Splunk has a very powerful parsing capability that allows you to describe whatever the shape of your line and the break. You would do this in the props.conf file. The confusion I think is that you appear to have data that is the result of a report on top (not events, nor is it a format for which you would use as a lookup) and on the bottom is something more along the lines of what you might use for a lookup. But ALL of it is "pipe" delimited. Each line would be broken with a carriage return and line feed ([\r\n]+) and you can choose to represent all of it in a number of ways. You are going to want to start here: Hopefully this will get you started... if not. Can you perhaps elaborate on your use case please? ... View more

@ajhstn same as above. you have to open up those macros and run each section, pipe by pipe to see why it's not running in your system. there is something silently failing. You have to reveal what it might be that is unique to your config... ... View more

oh gosh. sorry. upvoted somesoni2's comment. Funny, I just answered another question with that very fact... that special characters will usually need to be wrapped in appropriate quotes! Answer amended for posterity. ... View more

take a look at the doc. it's confusing... but the <<FIELD>> and <<MATCHSTR>> is literal... not a place holder. so: ...|foreach Total:* [eval <<MATCHSTR>> = <<FIELD>> + Yes:<<MATCHSTR>>] so when Total:AGUA and Yes:AGUA occur, for each Total:AGUA you get a new field called AGUA that is the sum of the original field, Total:AGUA and the other one... Yes:AGUA. ... View more

Is the field literally 'Total:BLAH' 'Yes:BLAH' or 'Total:FOO' 'Yes:FOO' or 'Total:BAR' 'Yes:BAR' ? because then it should work... that's what the wild cards are for. You may want to explore the data and test that the pattern of field NAMES is what you think they are... just to be sure... ... View more

understood. I expanded my answer for you. let me know if this helps. ... View more

Comparing the result of a search to lookup values is a very commonly used practice and you describe it perfectly. The way I phrased it is actually a clue as to how you perform this. You have a lookup table with valid hosts and users. (fields host, user) You have a search that retrieves successful logins and the hosts. To filter the search to show only those successful logins that occur on hosts NOT in your CSV you do this: yourSearchwith_host NOT [ inputlookup yourlookup |fields host ] so for instance you might say something like this: index=blah sourcetype=blah_foo host=* login_status=success NOT [inputlookup legit_user_host.csv | fields host] To really see what's going on, you'll want to look at the job inspector after it runs. to make sense of this, you can run each part of the search independently and imagine the 'venn diagram' the "NOT" creates. run index=blah sourcetype=blah_foo host=* login_status=success (any search that gives a list of the successful logins) then you can independently run |inputlookup legit_user_host.csv|fields host run it all together and Splunk will give you the results showing when the data in splunk does NOT match the lookup list. If you run the same and remove the NOT you get the list that DOES match... ... View more

I've amended my answer now that the question is clear. I'm going to also amend the title as the concept that one is a "clone" is irrelevant to the question. ... View more

george, If you are admin, and index=* doesn't show you the data, then you need to really consider whether the events are even getting to your indexer. You say you get a 200 OK but from what? If it works on one, and not on the other... consider that you might have the token wrong. Since the send is likely pretty frequent an error or warning would be easy to spot. take a look at the _internal index and see if you can spot something index=_internal sourcetype=splunkd NOT (log_level=INFO) check out the components ... View more

you are probably 'admin' on the dev box and admin can see everything. My colleague asks for the search syntax because if you don't specify the index you will be searching only over the main default index not a specific index. so if your search does not begin: index="nameofyourindex" then you probably aren't being shown the context of the index by default. you can either change that... or specify the name in the search. ... View more