I don't know that this will work any better, but you could try sourcetype=fortinet | dedup devname or sourcetype=fortinet | dedup devname | table devname ... View more

Is there any public documentation on SAP that one could use to write such an SAP ABAP app? ... View more

Where are the trace log files created? Are they actual files on disk, and are they in a binary or ASCII format? (Sorry, my SAP knowledge is limited) ... View more

You can use a wild card, and that's better than a separate monitor stanza for each file. If you need something more sophisticated, you can add either a whitelist or a blacklist to the spec. There is a new manual called "Getting Data In." Start here (http://www.splunk.com/base/Documentation/latest/Data/Configureyourinputs#Edit_inputs.conf) in the manual for help with inputs.conf ... View more

Actually, it should be easier than that! Splunk already knows about syslog format - a sourcetype for syslog is already included in Splunk. The definition for this sourcetype includes all the field extractions. The syslog-ng output should be in a syslog format. So what you really need to do is to tell Splunk that this input is sourcetype=syslog. If you set up the input using the Splunk Manager, you should be able to update it in the same way. You may need to restart Splunk for the changes to take effect. (You can also just add the line sourcetype=syslog to the appropriate stanza of inputs.conf, if you want to do this manually. You will also need to restart Splunk.) Now the bad news - changing the input sourcetype to syslog will only affect new data. If you can, remove any of the existing data from Splunk and re-index it. That's really the best way to fix the data. If you can't do that, then you may have to setup the field extractions; editing props.conf is one way to do that. Here are the instructions for setting up field extractions in the manuals. (Just to be clear: you definitely want search-time field extractions - and definitely do not want index-time field extractions.) ... View more

You can mask sensitive data at index time. (Ask more questions if that's not sufficient information!) However, once the data has been indexed, there is no way to change it. Not possible. All you can do it delete the data and re-index it. You can't mask it at search time. I know that isn't the answer that you wanted... sorry! ... View more

I don't think there is any way to parse XML prior to indexing. However, what might make your search run more quickly: does the value "mmsgw1.xxx.com" appear anywhere in the text other than the ActivityLogRecord_Common-ServerID? If this value does NOT appear anywhere else, then you could simply search index=test1 sourcetype="mmsgw" mmsgw1.xxx.com for example. Once you have retrieved only the needed events, then you could apply the xmlkvrecursive. This would probably be much faster, if you can eliminate a significant number of events in the initial search. And perhaps you don't need to parse the XML at all -- you only need to create the fields from the XML if you want to run searches, statistics or reports on the fields that are created by xmlkvrecursive. Based on the example event, the following search is also valid index=test1 sourcetype=mmsgw serverid="mmsgw1.xxx.com" userid="999999999" and also doesn't appear to require the XML to be parsed. But of course, I don't really understand the data... ... View more

If you delete a user, the PRIVATE saved searches owned by the user will be deleted. If the user has SHARED any searches (or other knowledge objects), they will still exist. The shared knowledge objects will still have the deleted user's name attached to them; they may be deleted by the Splunk admin. However, I don't think there is any mechanism for the Splunk admin to assign the saved searches to another user. ... View more

Nice! This appears to solve the problem neatly for the test. Just make sure that when you define the TCP input in Splunk, that you specify the sourcetype=syslog (You can select it from the list.) This will ensure that Splunk extracts the host name from the syslog events, rather than using the host name of the rsyslog server. ... View more

Splunk can monitor hundreds of files, and it's easy to set up if they are in the same directory tree. You might need to do some tuning after about 500 files - at that point, the default settings may not index the events as quickly as you'd like in Splunk. This is my preferred solution. Are the syslog messages defined as sourcetype=syslog in Splunk? Splunk's default processing for syslog messages is to extract the host name from the syslog event - this might fix the problem in the first paragraph. You can set the sourcetype as "from list" when you set up the input. ... View more

Q2 first: You can do 2 lookups in one search command, no problem. I don't think your syntax is quite right, though. And if you fix that, maybe it will answer both questions for you. Although you can do it all at once, I recommend that you set up your lookups in 3 steps: first, go to the Splunk Manager, then choose Lookups -> Lookup Table Files -> New. This is the place where you upload your .csv file to Splunk. Give it a filename.csv that you will use in subsequent commands. Second, in the Splunk Manager, choose Lookups -> Lookup Definitions -> New. This is the place where you actually name the lookup in Splunk - this name does not need to be the same as the filename.csv (and usually isn't). I'll call it mylookupName in the example below. Step 3 - Make sure you know the names of the fields. Your .csv file must have a first line that defines the column headings. For example, col1,col2,col3,col4 100,102,203,good 500,400,300,bad It is helpful, but not required, if the column names match the field names in your search - but let's assume that they don't. Your field names are field1, field2, field3 in your example above. Now, you can actually use the lookup in a search: mysearch | lookup mylookupName col1 as field1, col2 as field2, col3 as field3 OUTPUT col4 For each event, if there is a row in the .csv file where all the input values matches, there will be a resulting field, col4, available for that event. Also look at this question: Can I chain fields together between two different lookups? Finally, from your search, I wonder if what you want can be accomplished without a lookup command at all... can you give more details? ... View more

Is the ID unique within a log file? If so, the following should work... Assume that the ip address is extracted into a field named IP and that the transaction type is extracted as a field named transtype. I'd break this into 2 steps: First, associate an IP address with a source + ID combination sourcetype=yoursourcetype | transaction source, ID mvlist=true | eval reportIP = mvindex(IP,1) | table source, ID, reportIP, transtype This will give you a table output; notice that transtype will be a list of the various transaction types that appear in the transaction. Also note that we pick up only the first IP address that appears in the transaction. Our next task is to break this back into separate events, so we can count them... Count the number of each transaction types by IP - add this to the end of the previous search | mvexpand transtype | stats count by reportIP, transtype The full picture sourcetype=yoursourcetype | transaction source, ID mvlist=true | eval reportIP = mvindex(IP,1) | table source, ID, reportIP, transtype | mvexpand transtype | stats count by reportIP, transtype I hope this is what you wanted! Let me know if it doesn't work for you. ... View more

I wonder if it's because you have exceeded the maximum number of results. Take a look at the default file limits.conf in $SPLUNK_HOME/etc/system/default There should be a stanza in limits.conf that looks like this: [stats] maxresultrows = 50000 You can change this if you are exceeding the limit. Create a limits.conf file in $SPLUNK_HOME/etc/system/local that contains [stats] maxresultrows = 100000 Now see if your saved searches work as you expect. Another thing to think about - can you summarize the results in some way? It seems like a list of 70,000 events is not really that useful in and of itself, but then I don't know what you need to do with the list. ... View more

That is almost correct - they are handled in ASCII order, so watch out for case --upper case sorts ahead of lowercase. When order of processing is important, I always number the lookups. ... View more

I think you have been caught by way that stanzas in props.conf are processed; Splunk only makes one pass. You probably shouldn't count on the transformed sourcetype to be available for use in the second stanza. But there is an easy cure for your problem. You can eliminate the second stanza altogether, unless you already have some barracuda_sf events from some other input. [source::udp:514] TRANSFORMS-changesourcetype = set_sourcetype_barracuda_sf KV_MODE=none REPORT-bsf = bsf_scan, bsf_send, bsf_recv [barracuda_sf] KV_MODE=none REPORT-bsf = bsf_scan, bsf_send, bsf_recv ... View more

Try looking here for more info: Parsing pfSense Logs Part 2 Short answer: When setting up the input file, assign a manual sourcetype of pfSense Then include the following in props.conf [pfSense] SHOULD_LINEMERGE=true BREAK_ONLY_BEFORE=match You will probably have to define the fields yourself. There are a couple of ways to do that: Overview of Search-Time Field Extractions has an overview Create search-time field extractions by editing configuration files - this is my preferred way to do this Note: you do not want to use index-time extractions. ... View more

Can you login as one of the users, and then check to see if you can see the cached search results? My guess is that the saved results are private, and therefore the users can't read them. ... View more

See also http://splunk-base.splunk.com/answers/24807/what-changes-to-configuration-files-require-a-restart-of-splunk ... View more

This is a good start! ... View more

Ah, that's a more efficient solution. ... View more

Here are two examples of what you asked for -- Most number of methods used, not counting duplicates sourcetype="access_combined" | dedup clientip, method | top clientip Most number of unique pages visited sourcetype="access_combined" | dedup clientip, uri | top clientip But here are variations that might be more useful -- sourcetype="access_combined" | stats count by clientip, method | sort -count sourcetype="access_combined" | stats count by clientip, uri | sort -count Finally, take it to the next level by clicking the "Build Report" button and making a graph or chart of the results! ... View more