Hello I submit files with JSON-encoded lines to splunk, to a monitored directory. The fields are extracted correctly, except for the time which is not. My props.conf file: [nessusjson] INDEXED_EXTRACTIONS = json KV_MODE = none TIMESTAMP_FIELDS = 'N_scantime' TIME_FORMAT = '%s' MAX_TIMESTAMP_LOOKAHEAD = -1 I also tried KV_MODE = json . Below is an example of a JSON line, as seen by splunk. The EPOCH in N_scantime corresponds to Thu, 27 Nov 2014 09:17:08 GMT . 11/27/14 3:22:38.000 PM { [-] N_exploit: false N_exploit_malware: UNKNOWN N_exploit_metasploit: UNKNOWN N_exploit_metasploit_name: UNKNOWN N_nettype: wazaatype N_scantime: 1417079828 N_subnetname: wazaa name N_timeduration: 680 N_timeend: Thu Nov 27 14:33:58 2014 N_timeend_epoch: 1417098838 N_timestart: Thu Nov 27 14:22:38 2014 N_timestart_epoch: 1417098158 N_vendor: java } The idea was to have a specific field ( N_scantime ) in EPOCH format and other informational fields with the date ( N_time... ). I think I put in props.conf everything to indicate the field ( TIMESTAMP_FIELDS - I tried both with quotes and without), the format ( TIME_FORMAT - which is a 10 digits EPOCH in my case) and the fact that the filed can be anywhere ( MAX_TIMESTAMP_LOOKAHEAD ). I restarted the server for each test. It looks like the N_timestart field was parsed instead. Is there something I am still missing? Thank you for any pointers! ... View more

Hello I have two directories dir1 and dir2 monitored by splunk, new files in each directory are indexed, respectively, to index1 and index2 . I realized that I made a mistake and put myfile.txt in dir1 instead of dir2 . The events within were indexed to index1 , which is not a problem (I will deal with them later). I then moved the file via mv dir1/myfile.txt dir2 , the move was successful, I can now see myfile.txt in dir2 but the events are not visible in index2 . Other files which are in this directory are indexed correctly, so indexing as it works fine. Why moving the file did not trigger the indexing? From the standpoint of index2 these are new files and the fact that they have been previously indexed to index1 should not make any difference, right? ... View more

Hello Following up on a previous question about lookups I am looking for a way to either use or simulate wildcards in a .csv lookup file. I have fields like Microsoft Windows 8.1 Pro Microsoft Windows 8 Pro Microsoft Windows 7 Ultimate Microsoft Windows 7 Professional Microsoft Windows 7 Enterprise which I would like to group under, say, Windows Clients via a lookup. I do not know in advance what the values will be, so ideally I would like to be able to say Microsoft Windows 7*,Windows Client Microsoft Windows 8*,Windows Client which does not work as is (and was hinted so by aweitzman in his answer. is there a direct way to use regexp (or wildcards) in the lookup .csv file? The alternate solution I can think about would be to use an external script for the lookup which would get the field value and output something, based on a logic/algorithm within the script (as opposed to a csv) ... View more

Hello I have a search which reports a field N_os (a string indicating an Operating System). I wanted values from this field to be grouped together via a specific logic, the best I could find was a lookup. Follwing the docs I created my lookup file OSGroup.csv in /opt/splunk/etc/apps/MYAPP/lookups (tried both with double quotes and without): N_os,OSGroup "Microsoft Windows Server","Windows Server" "Microsoft Windows 7","Windows Client" "Microsoft Windows XP","Windows Client" "AIX 5.1","AIX" This file is referenced in /opt/splunk/etc/apps/MYAPP/local/transforms.conf : [OSGroup] filename = OSGroup.csv I restarted splunk. I then expected that a search ... | lookup OSGroup N_os OUTPUT OSGroup as N_osgroup creates a field N_osgroup . I do not know how the match is done (by substring, only if there is an exact match, ...) so I added one line to the csv file which contains a sample of the exact value of N_os (so that it matches - this is the "AIX" one). Unfortunaty the search proceeds as usual (no errors) but N_osgroup is not created on the left-side fields bar. Thank you in advance for any pointers! ... View more

Hello I have two fields field1 and field2 extracted from my search and I would like to present then in a table, with a string prepended to one of them, something like: field1 | http://example.com?field2 where http://example.com? is a fixed string. Is this possible? Thank you! ... View more

Hello It looks like the embedding features of splunk into a 3rd party app are gone from the docs in 6.x. What is then the supported way to get a chat into an external web page? Beyond the old answers to that question (includng mine) I saw that a possible solution would be with the SDK. When reading the doc, though, it looks like what is retreivable is pure data and not graphs (I would love to be wrong 🙂 If you have any examples...) Thank you! ... View more

Hello, I have a report with a stacked bar chart I would like to sort by "length", ie. by the total number of events in a bar. The search is ... | chart count over N_vendor by N_subnetname and I get the following graph. I would like to get the graph sorted so that the largest bars (java, redhat, ... in the graph) are up. Thanks! ... View more

Hello, I use splunk to index various sources, including files dropped into a directory and indexed to a given index. As of a sudden my files do not get indexed anymore. -- UPDATE -- The troubleshooting test described below (as INITIAL TROUBLESHOOTING) finally worked. I do not know why it took so much time to index (about an hour, usually indexes in minutes). This does not solve the initial problem though: I wanted to reindex data over a certain period. I did a index=myindex | delete over the period I wanted to reindex (90 days ago to now). This got rid of the data (at least on the search part). I reloaded the files in the tracked directory but the data did not reappear. I though that the cause might be that the source filenames are the same. So I renamed them (prefixing with a 0. ). Same thing: the new data does not reappear. So the problem now is not that the indexing of files in a directory does not work (good thing) but that I do not know how to force the reindexing on these new files (new = different filename, the contents will still match data indexed previously, but deleted as per above) Thanks for the help -- INITIAL TROUBLESHOOTING -- (this part now works, please see above) In order to investigate I created a brand new index and a brand new directory to host the files I want to drop. I took a few files which used to be indexed correctly, they are full of lines like Wed Aug 28 07:25:18 2013 N_hostip="10.103.43.253" N_netbios="UNKNOWN" N_dnsname="UNKNOWN" N_os="Linux Kernel 2.6.18-92cpx86_64 (x86_64)" N_pluginName="SSL Self-Signed Certificate" N_group="SSL" N_pluginID="57582" N_severity="2" N_risk="Medium" N_cvss="6.4" N_patch="UNKNOWN" N_dnt="0" N_subnetname="MHX" N_scanname="RECURRENT-Scheduled-003" N_vendor="ssl" N_product="UNKNOWN" and dropped them into that directory. They are not visible in splunk The index in Manager is seen as empty (no events, 1 MB size) The directory I use, as seen in Manager (Data Inputs) shows 49 files, which is correct There is ample disk space on the machine splunk works as it (I can search etc.) I searched for "All time" -- even though the events are max 90 days old I even restarted splunk for good measure The license is OK (50 MB out of 1 GB today, the files are a few MB) I would appreciate any help on what to test now to get these data in, before I open a ticket (I hope I missed something obvious bo no idea where) Thanks! ... View more