More odd behavior. With the configuration recently posted, if I run the Splunk search: index=foo | sort 0 _time and use the GUI Splunk web based time identifier interface (the button) to say "Last 5 mins", the search will search through only 1,000,000 of the 8mil + events (better result) When I used: index=foo earliest=-5m | sort 0 _time It will find the last 5 min of events, get to ~1,000,000 and then the search hangs... The good news is I think its using the changes to indexes.conf somewhat correctly because when I test to see how many events I have for the folder for today only, its about 1,000,000. So I think the behavior is that it is searching through only this one day today... I assume if I want the search to be even quicker, I'd have to sort my subdirectories further and have them into years/months/days/hours/minutes which I'm not sure we're willing to do. I'm not sure why its hanging when we use the earliest keyword but your answer solved my problem. ... View more

Yes, I've been restarting Splunk after I edit it as well. ... View more

Yeah I cannot understand why when I run this query: index="juniper" earliest=-5m | sort 0 _time It still finds the last 5 mins and then continues searching through the rest of directories... This is my indexes.conf now: vix.command.arg.3 = $SPLUNK_HOME/bin/jars/SplunkMR-hy2.jar vix.env.HADOOP_HOME = /usr/hdp/2.5.0.0-1245/hadoop vix.env.HUNK_THIRDPARTY_JARS = $SPLUNK_HOME/bin/jars/thirdparty/common/avro-1.7.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/avro-mapred-1.7.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/commons-compress-1.10.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/commons-io-2.4.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/libfb303-0.9.2.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/parquet-hive-bundle-1.6.0.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/snappy-java-1.1.1.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive_1_2/hive-exec-1.2.1.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive_1_2/hive-metastore-1.2.1.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive_1_2/hive-serde-1.2.1.jar vix.env.JAVA_HOME = /usr/lib/jvm/jre-1.8.0 vix.family = hadoop vix.fs.default.name = hdfs://10.x.x.x:xxxx vix.mapreduce.framework.name = yarn vix.output.buckets.max.network.bandwidth = 0 vix.splunk.home.hdfs = /tmp/splunk vix.yarn.resourcemanager.address = hdfs://10.x.x.x:xxxx [juniper] vix.provider = hadoopoly vix.input.1.path = /topics/foo/... vix.input.1.et.regex = /topics/foo/(\d\d\d\d)/(\d\d)/(\d\d)/.+ vix.input.1.et.format = yyyyMMdd vix.input.1.et.offset = 0 vix.input.1.lt.regex = /topics/foo/(\d\d\d\d)/(\d\d)/(\d\d)/.+ vix.input.1.lt.format = yyyyMMdd vix.input.1.lt.offset = 86400 ... View more

As a side note to all who are helping (thank you by the way), I have changed our directory structure to the recommended /topic/foo/year/month/day instead of /topic/foo/month-day-year and am altering the indexes.conf accordingly. Hopefully, this more detailed and partitioned by time structure will allow for the regex to work properly (I changed that as well to: vix.input.1.et.regex = /topics/foo/(\d\d\d\d)/(\d\d)/(\d\d)/.+ vix.input.1.et.format = yyyyMMdd ... View more

Yup, I figured that out - thanks for the heads up. ... View more

Our data is stored: /topics/foo/01-12-2017/filename.log Trying to use these parameters to indicate earliest time but seems to not be working. vix.input.1.et.regex = /topics/firewall/(\d+)-(\d+)-(\d+) vix.input.1.et.format = MMddyyyy ... View more

Thanks for the insight. I tried to change my indexes.conf accordingly below and it still searches throughout all the files. Perhaps my regex or wording is wrong. The file structure path of I'm using is /topics/foo/01-12-2017/ This is what I added to indexex.conf: vix.input.1.et.regex = /topics/foo/(\d+)-(\d+)-(\d+) vix.input.1.et.format = MMddyyyy Then I ran this query: index=foo earliest=-5m | sort 0 _time And unfortunately it still ran through all the files before finishing the search. Any ideas? ... View more

[provider:XXX] vix.command.arg.3 = $SPLUNK_HOME/bin/jars/SplunkMR-hy2.jar vix.env.HADOOP_HOME = /usr/hdp/2.5.0.0-1245/hadoop vix.env.HUNK_THIRDPARTY_JARS = $SPLUNK_HOME/bin/jars/thirdparty/common/avro-1.7.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/avro-mapred-1.7.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/commons-compress-1.10.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/commons-io-2.4.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/libfb303-0.9.2.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/parquet-hive-bundle-1.6.0.jar,$SPLUNK_HOME/bin/jars/thirdparty/common/snappy-java-1.1.1.7.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive_1_2/hive-exec-1.2.1.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive_1_2/hive-metastore-1.2.1.jar,$SPLUNK_HOME/bin/jars/thirdparty/hive_1_2/hive-serde-1.2.1.jar vix.env.JAVA_HOME = /usr/lib/jvm/jre-1.8.0 vix.family = hadoop vix.fs.default.name = hdfs://10.x.x.x.:xxxx vix.mapreduce.framework.name = yarn vix.output.buckets.max.network.bandwidth = 0 vix.splunk.home.hdfs = /tmp/splunk vix.yarn.resourcemanager.address = hdfs://10.x.x.x:xxxx [juniper] vix.input.1.path = /topics/firewall/... vix.provider = XXX ... View more

I don't actually see those properties : vix.input.* when browsing the properties listed under Additional Settings in Virtual Indexes in Splunk Web. Are these properties somewhere else or do we need to add them and what should they be set to? ... View more

Yeah it does, you are correct. As soon as I add the transaction line, it returns no results... which is mind-boggling to me. I add this line and it returned no results: | transaction host startswith="ALPXX=re-connecting" endswith="ALPXX=online" The more I think about it, taking the query results with the append and not with the transaction does return only two events, the first re-connecting and the last online for ALPXX. Running a transaction on those two events would maybe at least return one transaction event for those two events but even if it did that, that's not ultimately what I want. I'm feeling pretty stumped on this. I have this working via a Python SDK which outputs to an csv but I need to create a dashboard with visualizations for it. I'm considering taking the csv instead and doing an input lookup on it to get the visualizations. I feel like I understand what transactions and concurrency does but for some reason it skips over necessary stopping points. Sometimes I wish that we could inject hybrid code into SPL kinda like PL-SQL. ... View more

As a side note, when I took out this line: | dedup consecutive=true keepempty=true host sortby +_time I also had to do the search for ALPXX=* instead of ALP='re-connecting' or it wouldn't return any results. Im very curious to hear of the nature of how transaction processes the logs. Thanks for any guidance. ... View more

Thank you for replying. For some reason your solution causes the query to return no results. It is baffling to me that transaction searches that way. Does it start at least at the earliest time and then proceed forward searching for the endswith and then step back to find the paired startswith to create a transaction? When I took out this line: | dedup consecutive=true keepempty=true host sortby +_time It did return results but they seem to skip even more logs till it gets to the one it flaga as the end. Its baffling cause I am looking at the results of which ones it picks to be the endswith and it chooses one that is surrounded by ALPXX=online log entries, so its netiher the latest or earliest in that specific transaction time. I feel like I'm missing something pivotal that will be obvious once I see it. Thanks for your input. I am going to manipulate your query and see if I can figure out how to put it to my advantage. ... View more

Thanks for all your help. That worked like a charm. ... View more

Thank you again for your answer. Is there a way to put it into a timechart span=1d? I have tried to convert it to stats latest(requests) as latestRequests by host | timechart span=1d sum(latestRequests) as Total but that doesnt return any results. Also tried adding timechart at the end after the second stat I guess stats and timechart can't be mixed... so I'd have to find a way to do the stats functionality with timechart? ... View more

Thank you so much! Worked perfectly. I knew it had something to do with multiple instances of stats. ... View more

Update to this question... I discovered the problem was that I was printing the results to the console for debugging and the actual search finished much sooner than the printing finished and by the time the printing was done, the job had expired, I guess. Once I ran the job and the parsing into proper data structures afterward, it worked fine. I resolved the need for a debug printing by running the job a second time after the debug printing was done .. not the most efficient solution. Any other possible solutions would be welcome. ... View more

Thank you yannK. ... View more

Thank you for your answer. It clarified whether it is possible or not. Unfortunately, you are right, using the query itself defeating the purpose of getting the acceleration benefits and bypass the use of the pivot UI completely. I have a list of questions/concerns about the data models/pivot system they have in place that I am taking with me to Splunk conference. ... View more

Here is an example of the query I am trying to do: sourcetype=xxx-yyy FOO | join host [search sourcetype=xxx-yyy BAR] | eval var=(ABC - (DEF + GHI + FOO + BAR)) | timechart span=30m sum(var) by host Im trying to divide the columns up by the host so not sure how to use "by _time" with this one in data models and generate a pivot from it. Thanks for all your help. ... View more