Word of caution on this one. I am a great fan of the DS, but I hadn't tried pushing indexer confs until yesterday. Unfortunately I was a little bit lazy when configuring the indexes.conf going out as part of the app; "No need to specify hot/warm/cold/thawed paths, just go with the default"... WRONG! Since the indexer did not know where to store the indexes, it promptly died on startup, making the DS useless for fixing the problem. So I had to go out to each indexer and manually delete the bad indexer-app (after correctly defining it on the DS, of course). You learn a little each day. /k ... View more

Encountered a similar problem... To fix, I re-wrote the query with spaces instead of tabs and it appears to have worked. For example, I initially had: index="searchindex" sourcetype="sourcetype" AND thisandthat | test1 | test2 | append [search blabalbal | blablabla | blablablabla | blabla] | chart blablaa Fixed by changing it to: index="searchindex" sourcetype="sourcetype" AND thisandthat | test1 | test2 | append [search blabalbal | blablabla | blablablabla | blabla] | chart blablaa I wrote the initial query in notepad++, copied/pasted it which is what failed. When re-written without the tabs it was fine. I assume there is a character it didn't like that was inserted by NP++. ... View more

You could replace newline characters using rex : <yoursearch> | rex field=_raw mode=sed "s/[\r\n]//g" The default output in the search app will still wrap lines though, so if you want everything unwrapped on one single line one option would be to use table . <yoursearch> | eval eventtext=_raw | table _time eventtext (using _raw directly in table does not work). ... View more

You could build transactions using MYFIELD to tie them together and then check which ones only contain one event and count those. <yourbasesearch> | transaction MYFIELD | where eventcount=1 | stats count ... View more

Hi, I have been struggling with this for a long time. Thanks a lot. I am trying to display events from the past in the same graph as current events in a graph (Today vs last week). ... View more

I can think of two ways to achieve this. Single out the recurring values and dedup them out. For instance if the duplicate events happen within the same minute and your search is covering the last hours, you could use dedup date_hour date_minute date_second . This will obviously not be failsafe though, so might not be the best option. Instead of where , use search to find transactions that have both values as sourcetypes. sourcetype=AS_CDR OR sourcetype=MSP-PROD |dedup _raw |eval CID1=coalesce(AS_Call_ID,MSP_Call_ID) |transaction fields=CID1 maxspan=1m keepevicted=true |search eventcount>1 AND sourcetype=AS_CDR AND sourcetype=MSP-PROD ... View more

I did this before I found what Luke Harris mentioned above: link text It requires Perl and a standard Splunk installation (might work with a forwarder, I don't know). Troy ... View more

I have been told that we use strftime for time formating and for reference in props.conf http://php.net/manual/en/function.strftime.php Based on your string - 15:09:55.128 UTC AUG 31 2011 Try this %H:%M:%S.%3N UTC %h %d %Y You also need to set the TIME_PREFIX = ^\d+?\s\w{4}:\s+?[ and possibly MAX_TIMESTAMP_LOOKAHEAD = 35 (number of characters to look into the line to find the timestamp The day depends – is it 1-31 = %e or 01-31 = %d According to strftime - %t is a tab, I think you just have spaces. ... View more

Makes sense! The only command that seems not to be working is isnotnull(field) but I can do without that for now. Thank you! ... View more

just wanted to compliment you on an extremely well-written question. thanks! ... View more

Worked wonderfully, thank you! ... View more

Summary indexes are not a sort of excel file in which you populate some cells of the same lines at different times. Deduping works at the "row" level, not column. However, it all depends on the search you use to read summary data and present it to the user, so fields on different summary index events might become part of the same row in a table (transactions, | bucket _time span=1h | stats first(field1), first(field6) by _time, ....) I think you might want to create a test summary index and test this a little bit. I'm not sure I got your usecase right ... View more

The documentation for version 4.2.1 does not show convert as deprecated. Just curious if you're going by best practices or is the function in fact deprecated and the documentation doesn't reflect that? ... View more

index=correlation sourcetype=source1 OR sourcetype=source2 | eval key_field=if(sourcetype == 'source1', egress_call_id_from_source1, ingress_id_for_source2) | timechart count by key_field ... View more

Awesome, thank you so much! ... View more

Hm, so it does look like it will be a massive regex. What sort of overhead would a regex of that size incur? Sadly one issue we'd run into with this is the fact that our source moves multiple times. ... View more

Just tried it using eventstats, works like a bell. Thank you! I'll have to play with this a bit more. ... View more

The transaction command cannot do the correlation you're looking for - however here's a way to do it: create a field that unifies all the records. In the following simplistic example I am assuming that you can somehow differentiate between local and non-local caller ids, let's say local caller ids start with 12, then the following search will generate the transactions you're looking for .... | eval cid=if(match(LocalCallID, "^12"), LocalCallID, RelatedCallID) | transaction maxspan=10s cid ... View more

Splunk's search language is extremely flexible. There are a number of commands out-of-the-box such as "| eval" and "| transaction". If you find something that can't be done using a shipped command, Splunk gives the ability to create your own search command using python. See: Search Reference ... View more

The v4.2 Interactive Field Extractor sucks IMO. Highly recommend using something like RegEx Buddy or RegEx Magic. They are cheap apps but really make short work of regex's. ... View more