We have successfully created and deployed an application.
We are currently attempting to consume json data written to a file system on universal forwarder.
1) we created - to
/var/log/github_api/
2) we placed some test json files in there.
(named .json, and .txt, as well as no file extention).
[monitor:///var/log/github_api/]
index=github_api
ignoreOlderThan=1d
host_segment = 3
sourcetype=json
3) used deploy server to push out configs.
Result: Only getting some files but not all.
===============Begin TSHOOT==============
splunkd.log information.
03-06-2013 15:04:39.782 -0500 INFO DeployedApplication - Refreshed app: ise_git_phlow_inputs for service class: syslog from archive: /opt/splunkforwarder/var/run/syslog/ise_git_phlow_inputs-1362598641.bundle
03-06-2013 15:04:40.083 -0500 INFO TailingProcessor - Parsing configuration stanza: monitor:///var/log/github_api/.
Seems to be liking the config.
=========== Next
Looks like we are getting no events into the newly created index.
```
03-06-2013 15:04:39.782 -0500 INFO DeployedApplication - Refreshed app: ise_git_phlow_inputs for service class: syslog from archive: /opt/splunkforwarder/var/run/syslog/ise_git_phlow_inputs-1362598641.bundle
03-06-2013 15:04:40.083 -0500 INFO TailingProcessor - Parsing configuration stanza: monitor:///var/log/github_api/.
```
github_api
500,000
None
1
0
N/A
N/A
/data/hotwarm-indexes/github_api/db
ise_all_indexer_base
Enabled | Disable Delete
====== NEXT UP
Another update. looking into the indexer splunkd.logs for anything relevant
03-06-2013 14:09:52.945 -0500 INFO HotDBManager - idx=github_api Setting hot mgr params: maxHotSpanSecs=7776000 snapBucketTimespans=false maxHotBuckets=3 maxDataSizeBytes=786432000 quarantinePastSecs=77760000 quarantineFutureSecs=2592000
03-06-2013 14:09:52.945 -0500 INFO databasePartitionPolicy - idx=github_api Initialized with params='[300,60,188697600,,,,786432000,5,true,500000,5,5,false,3,0,_blocksignature,7776000,1000000,0,3,77760000,2592000,131072,25,0,15,0,0,-1,18446744073709551615,2592000,true,60000,300000,false]' isSlave=false needApplyDeleteJournal=false
03-06-2013 14:09:52.945 -0500 INFO DatabaseDirectoryManager - Writing a bucket manifest in hotWarmPath='/data/hotwarm-indexes/github_api/db'. Reason='Refreshing manifest.'
03-06-2013 14:09:52.946 -0500 INFO databasePartitionPolicy - openDatabases complete currentId=0 idx=github_api
[root@ic-spk01 splunk]#
====== Another update
Main indexer - suspicious.....
So this could have something to do with it:
03-06-2013 15:35:40.563 -0500 WARN DateParserVerbose - Accepted time (Fri Jan 25 10:50:12 2013) is suspiciously far away from the previous event's time (Mon Jan 28 16:31:36 2013), but still accepted because it was extracted by the same pattern. Context: source::/var/log/github_api/test2.txt|host::github_api|json|remoteport::58235
====== FINAL ANSWER?
So as part of the troubleshooting effort we did the following:
1) copied the existing sample JSON data file and made a replica
cp -r notifications test1.txt
cp -r test1.txt test.json
cp -r test1.txt test2.txt
Upon indexing?
on Indexer (splunkd.log)
03-06-2013 15:35:40.563 -0500 WARN DateParserVerbose - Accepted time (Sat Jan 26 09:33:46 2013) is suspiciously far away from the previous event's time (Thu Jan 31 10:11:27 2013), but still accepted because it was extracted by the same pattern. Context: source::/var/log/github_api/test2.txt|host::github_api|json|remoteport::58235
03-06-2013 15:35:40.563 -0500 WARN DateParserVerbose - Accepted time (Fri Jan 25 10:50:12 2013) is suspiciously far away from the previous event's time (Mon Jan 28 16:31:36 2013), but still accepted because it was extracted by the same pattern. Context: source::/var/log/github_api/test2.txt|host::github_api|json|remoteport::58235
03-06-2013 15:35:40.817 -0500 INFO databasePartitionPolicy - idx=github_api Creating hot bucket=hot_v1_1, given event timestamped=1354488490
03-06-2013 15:35:40.817 -0500 INFO DatabaseDirectoryManager - Writing a bucket manifest in hotWarmPath='/data/hotwarm-indexes/github_api/db'. Reason='Bucket directory structure changed.'
on Universal forwarder (splunkd.log)
03-06-2013 15:25:29.740 -0500 INFO TailingProcessor - Parsing configuration stanza: monitor:///var/log/github_api/.
... View more