Getting Data In

How come when I try to add data, there is no ingestion is happening?

splunkannm
New Member

Im trying to use the index once option of add data to ingest a 6G tsv file. It does not show any preview and does not ingest data though it says successful ( 0 events ).

it has 4.5 million records.

0 Karma

mstjohn_splunk
Splunk Employee
Splunk Employee

Hi @splunkannm,

Did the answer below solve your problem? If so, please resolve this post by approving it! If your problem is still not solved, keep us updated so that someone else can help ya. Thanks for posting!

0 Karma

MuS
SplunkTrust
SplunkTrust

Hi splunkannm,

if you have a large amount of data to index once, put it into $SPLUNK_HOME/var/spool/splunk/ this is a sinkhole directory. Anything you put in there will be indexed, and once indexed it will be deleted by Splunk.
The events will be available in index=main or whatever you did setup as the default index.

More details in the docs here http://docs.splunk.com/Documentation/Splunk/latest/Data/MonitorFilesandDirectories#Why_use_upload_or...

If you need to specify a different index or sourcetype, simply create another sinkhole directory using inputs.conf and the [batch://...] stanza http://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf#BATCH_.28.22Upload_a_file.22_in_...

Hope this helps ...

cheers, MuS

0 Karma

splunkannm
New Member

While this is useful information , that I will keep note of, my question is why is the add data option not working and throwing a JS error (404). I even tried a simple file as below :

a,b,c
1,2,3

0 Karma

pruthvikrishnap
Contributor

Hi,
Try selecting a suitable sourcetype while indexing the data.
Also as adonio mentioned, please try indexing a file which has less volume and check if everything works fine.

0 Karma

splunkannm
New Member

I tried with a simple one liner file as mentioned in my comment above. Its not working still..

0 Karma

splunkannm
New Member

Actually its more like -> add data even a one line csv is not working . Getting this JS error

splunkd/_raw/services/dmc-conf/settings/settings?output_mode=json&=1539635024124 404 not found

0 Karma

adonio
Ultra Champion

the single file upload is limited to 500MB
try and monitor the file:
http://docs.splunk.com/Documentation/Splunk/7.2.0/Data/Monitorfilesanddirectories

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...