We have a Python script that just generates some dummy log files in order to test ingestion, we are using MapR and Splunk. Every once in awhile we get an error saying Out of Memory when reading a very large line... which I thought was unusual since the events being put into the log file by the Python script are pretty small. Then I found the offending line... It looks like this ^@^@^@^@^@^@^@^@^@^@^@ And repeats for 2 million characters. In the one instances I found it it was at the beginning of a file that had just rolled over Does anyone know what these symbols mean? Why would they be being put into our log file? ... View more

We are currently using HUNK and MapR to read in some dummy data which is basically a collection of repeating events. We are trying to test the data ingestion of MapR. In my script, at first, I had the event divided by one newline. Then this error happened. So I added a new newline thinking that maybe a few events were being concatenated. But the error still came back. The question is.. how do we avoid seeing this error? It says we can set mapreduce.input.linrecordreader.line.maxlength to a lower value but as far as I can see the only place that is located is in a java file. Is there an xml properties files this is located in I can change? Anyone else had an error like this? IOException - Out of memory error while reading a very large single line input record. To skip this record set mapreduce.input.linerecordreader.line.maxlength to a lower value. Current value: 2147483647, jvm heap size: 508035072, potential value: 31752192 ... View more

We have the MapR filesystem and Hunk on the same node, have set up a Provider and Virtual and Im getting an error when trying to run a query other than a basic index search. "index=test" produces correct results "index=test keyword" produces the error below Also see screenshot attachments of Provider and VI config. Error from search.log: 05-23-2018 20:49:13.581 ERROR ERP.maproly - Caused by: java.lang.RuntimeException: summary_id did not exist in search info: {_tz=### SERIALIZED TIMEZONE FORMAT 1.0;C0;Y0 NW 55 54 43;$, now=1527108550.000000000, _sid=1527108550.15, site=default, _api_et=1527019200.000000000, _api_lt=1527108550.000000000, _dsi_id=0, _keySet=dghsd index::test1, _ppc.bs=$SPLUNK_ETC, _search=search index=test1 dghsd, _shp_id=C0F05B71-38F6-4B2F-ACF7-5756DCD4CAB6, _endTime=1527108550.000000000, _ppc.app=search, read_raw=1, realtime=0, _countMap=duration.command.search.expand_search;39;duration.command.search.parse_directives;0;duration.dispatch.evaluate.search;54;duration.startup.configuration;11;duration.startup.handoff;3;invocations.command.search.expand_search;1;invocations.command.search.parse_directives;1;invocations.dispatch.evaluate.search;1;invocations.startup.configuration;1;invocations.startup.handoff;1;, _ppc.user=admin, check_dangerous_command=0, _default_group=, generation_id=0, _bundle_version=0, indexed_realtime=0, search_can_be_event_type=1, indexed_realtime_offset=0, kv_store_settings=hosts;127.0.0.1:8191\;;local;127.0.0.1:8191;read_preference;C0F05B71-38F6-4B2F-ACF7-5756DCD4CAB6;replica_set_name;C0F05B71-38F6-4B2F-ACF7-5756DCD4CAB6;status;ready;, _timeline_events_preview=0, is_cluster_slave=0, internal_only=0, is_batch_mode=0, _remote_search=search (index=test1 dghsd) | fields keepcolorder=t "" "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server", summary_stopped=0, _search_metrics={"ConsideredBuckets":0,"EliminatedBuckets":0,"ConsideredEvents":0,"TotalSlicesInBuckets":0,"DecompressedSlices":0,"FieldMetadata_Events":"","Partition":{}}, _is_summary_index=0, _search_StartUp_Spent=0, _is_keepalive=0, _is_scheduled=0, _splunkd_port=8089, _is_export=0, _is_remote=0, _maxevents=0, _search_et=1527019200.000000000, _search_lt=1527108550.000000000, _startTime=1527019200.000000000, _timestamp=1527108550.251636000, is_saved_search=0, is_remote_sorted=0, _search_StartTime=1527108550.250084000, remote_log_download_mode=disabledSavedSearches, kv_store_additional_settings=hosts_guids;C0F05B71-38F6-4B2F-ACF7-5756DCD4CAB6\;;, _rt_batch_retry=0, _auth_token=8cntqHuq0Rb0Lz3T^YcThKI7mBeHBy4ki7SPCQHDHCuMQq1haa4BENOHDqd43diGvYDkRlyNuR6xs1eUwYfPE4PBO1IeTwbkxIAG2JxOpUIpE^IOBBwklXUWaqa, _drop_count=0, _provenance=UI:Search, _scan_count=0, is_shc_mode=0, rt_backfill=0, sample_seed=0, _bs_thread_count=1, _retry_count=0, _splunkd_uri=https://127.0.0.1:8089, replay_speed=0, _exported_results=0, sample_ratio=1, summary_mode=none, _query_finished=1, _optional_fields_json={}, enable_event_stream=1, _splunkd_protocol=https, _read_buckets_since_startup=0, _bs_pipeline_identifier=0, _request_finalization=0} 05-23-2018 20:49:13.581 ERROR ERP.maproly - at com.splunk.mr.SplunkMR.getSummaryId(SplunkMR.java:507) 05-23-2018 20:49:13.581 ERROR ERP.maproly - at com.splunk.mr.SplunkMR$SearchHandler.executeMapReduce(SplunkMR.java:1359) 05-23-2018 20:49:13.581 ERROR ERP.maproly - at com.splunk.mr.SplunkMR$SearchHandler.executeImpl(SplunkMR.java:1067) 05-23-2018 20:49:13.581 ERROR ERP.maproly - at com.splunk.mr.SplunkMR$SearchHandler.execute(SplunkMR.java:906) 05-23-2018 20:49:13.581 ERROR ERP.maproly - at com.splunk.mr.SplunkMR.runImpl(SplunkMR.java:1802) 05-23-2018 20:49:13.581 ERROR ERP.maproly - at com.splunk.mr.SplunkMR.run(SplunkMR.java:1551) 05-23-2018 20:49:13.581 ERROR ERP.maproly - ... 3 more 05-23-2018 20:49:13.597 INFO ERP.maproly - SplunkMR - finishing, version=6.2 ... 05-23-2018 20:49:13.597 INFO ERP.maproly - DispatchReaper - Skip dispatch reaping, top level HDFS dispatch dir=/user/root/splunk/splunkmr/dispatch does not exist. 05-23-2018 20:49:13.621 ERROR ERP.maproly - Error while invoking command: /opt/mapr/hadoop/hadoop-2.7.0/bin/hadoop com.splunk.mr.SplunkMR - Return code: 255 ... View more

So we have this query: index=_internal type=Usage st!=splunk_metrics earliest=-1d@d latest=-0d@d | bucket _time span=1d | stats sum(eval(b/1024/1024/1024)) as GB by _time Its been running on Splunk for years for us, producing some info about how much is being indexed per day .. we upgraded to Splunk 6.6 and it seems like it doesn't work anymore. I don't see the field "type" anymore Does anyone know if they changed this in this new version? ... View more