Hi Lisa, I have the same problem too in Splunk 6.1, as many others, for a quite important prospect. I also had as last resort the idea of adding at the beginning of the _raw data the timestamp extracted from the source file, with date and time of the generation of the informations. I only have a doubt: isn't timestamp assigned during the parsing phase before the Custom configurations in props.conf, like transforms and so on? We tried that but with no results... Regards, Marco ... View more

Thanks Matthieu, I also implemented ES and was fine. I'm not working on that project any more, and I remember that the logs were tagged following CIM, otherwise the Correlational Rule doesn't recognize them and apply. Marco ... View more

It this answers your question, don't forget to mark it as valid here on answers.com! And happy Splunking! Marco ... View more

Glad it worked! Please don't forget to mark the question answered! And happy Splunking! Marco ... View more

to be honest I'm quite busy and couldn't check carefully your searches if there's some other mistakes. Sometimes, bwt, _time field gives problem with substitutions or other operation. That's why I suggested a cleaner approach from the beginning, with the correct time extraction. Let me know! Marco ... View more

yYu can design a saved search that extract the data you need from the main index and add them to the summary index for a fixed time-frame (1hour for instance) and set it to be scheduled each hour. Then you have the "fill_summary_index.py" python script that allows you to run back in time the search for the desired period of time. For details, see http://docs.splunk.com/Documentation/Splunk/6.0.3/Knowledge/Managesummaryindexgapsandoverlaps#Use_the_backfill_script_to_add_other_data_or_fill_summary_index_gaps Marco ... View more

Had the same problem and solved it following aelliott's suggestion. I made time ago a copy of the search app but with another name. then there was another "default/data/ui/manager" dir in the copied app. Just deleted that dir and Splunk 6 worked fine! No more duplicates!!! Check if under $SPLUNK_HOME/etc/apps/* you have another "manager" dir somewhere. Marco ... View more

What happened with the App?! It's not on SplunkBase anymore! Do you plan a new release or some access policy in Facebook has blocked access to his data? Marco ... View more

Will4t, From your description, I thing you just need a regular lookup, nit a Join... What is still not clear to me, is why you need to create a new index C: do you need (and why) to store the result of the lookup somewhere? Marco ... View more

Thanks for the hint, but it doesn't apply, because it's the same field in the same position that's used as a queued_as information. It's a sort of inner join problem actually.... ... View more

good job. in that way you have the different login for each different user. My stats was counting all events where the field user had a value. Marco ... View more

sorry folks for some confusion.. I was writing from my iPad and there was some extra capitalization due to autocorrect.... Marco ... View more

And remember that you can write a single "Rex" matching all the different fields you need to extract, if of course are ALL present in the same event... Marco ... View more

You should have been added TIME_PREFIX=^ ^ meas "start of line". ... View more

I had a similar problem due to the first 260 chars in the file being alway the same due to long headers. I solved this in the inputs.conf like this: [monitor:///........./appdir/SD*.ERR_*.Z] disabled = false followTail = 0 sourcetype = my_sourcetype initCrcLength = 330 crcSalt = <SOURCE> In my case, we had thousands of file being written in the same "appdir" and severa times the "ERR" files were skipped because of same headers. Marco ... View more

beware that this option is valid only for a stanza like [source::filename] ... View more

You can override the 50.000 default maximum number of events returned from the join subsearch in limits.conf using [join] subsearch_maxout = <yourvalue> Marco ... View more

Look, this is an example I just did friday for a Customer: sourcetype="sap_fea" IVN=* id_elab =* | join type=outer id_elab,IVN [search sourcetype=sap_err ] | fillnull value=OK message | table _time, id_elab, IVN, success, message where I had to merge data from sap_fea sourcetype contaning id_elab, IVN and success fields, with the "sap_err" sourcetype where, if there's and error in sap_fea, in the "message" fields there's the error description. If there's no error, I have no corresponding event in sourcetype "sap_err" and that's why I used outer join and "fillnull". And it works! 🙂 Marco ... View more

First of all, you will see events just from your 'default' indexes, depending on the User's role, as someoni2 said. Then, if you have strange dates as newest or late see bent, this means that there's a time stamp problem: so if you see events in the future, check for events in that time range and check of time stamp in the event is translated. The same if you have events older than they are supposed to be. It's a quite common problem, if Imunderstood right your question. Happy Splunking, Marco ... View more

Correct answer should be linu1988s. index=A x=* | rename x as b | join b [ search index=B b=*] | table a, b, c, y I added x=* and b= * to be sure to extract events with fields x and b with some value. Marco ... View more

and for the empty fields you can use the "fillnull" command. ... View more

Check this I just wrote on LinkedIn: Hi! I don't know how many of you had to manage data coming from logs with fixed-lenght records. This is the old cobol/mainframe like method of managing datafiles. With Splunk there's no problem in defining field extraction with regex. If, for instance we have a schema like this: field1 - 4chars field2 - 8chars and so on, we could write a simple EXTRACT like this: [mysourcetype] EXTRACT = ^(? .{4})(? .{8}) and so on.... This works fine in a search. So you can do something like: * | stats count by field1 But, what happens if you try to filter by the field value? If you try something like * field1=abcd this won't work!!! To make it work, you have to write something like * field1=abcd So, all the power of fields become useless with this kind of data! Transactions are gone, subsearches are gone, most of the power of Splunk Analytics is gone! Gone forever? Thanks god not! I found a very easy workaround to this. I know it's not a real solution to the problem, but, if you don't have a strong requirement on file integrity, this will save your life!!! The answer is: "Add fields delimiters!!!" Splunk indexing works by indexing every single "string", which means every sequence of letters and numbers divided by some non alphanumeric character, like ".,!/!?:" and so on. So, the solution is to modify the raw data written in the Index using such a separator. Here is how: 1. props.conf [mysourcetype] TRANSFORMS = add_separators EXTRACT = ^(? .{4}).(? .{8}). and so on.... transforms.conf [add_separators] DEST_KEY = _raw SOURCE_KEY = _raw REGEX = ^(.{4})(.8{})(.*) FORMAT = $1.$2.$3 Now, finally, everything works back as usual!! ... View more

DESK_KEY = timestamp does not exist. DESK_KEY = _time is fine, but _time must be in EPOC time.... That's my problem too. ... View more