topic I cant get Splunk to read Boss Of SOC v1 Data Set in Getting Data In

I cant get Splunk to read Boss Of SOC v1 Data Set

dablab — Sat, 18 Nov 2023 08:32:26 GMT

Hey! So Im using an EC2 splunk ami and have all the correct apps loaded but cannot for the life of me get the boss v1 data in my environment.

I've put it into $SPLUNK_HOME/etc/apps (as mentioned in github) and it did not work, it simply does not pick up that this is a data set and instead is comfortably in my apps.

Loading it in other ways means it doesnt come through correctly. Is this a timestamp issue?

Any help would be so appreciated

Re: I cant get Splunk to read Boss Of SOC v1 Data Set

richgalloway — Sat, 18 Nov 2023 13:08:46 GMT

Something's not right in that screenshot. The contents of indexes.conf should not be indexed. I suspect some instructions are being misinterpreted.

Please tell us more details about how you are trying to load the data. Provide the exact steps followed or a link to them.

Re: I cant get Splunk to read Boss Of SOC v1 Data Set

PickleRick — Sat, 18 Nov 2023 13:37:29 GMT

+1 on @richgalloway 's doubt

You are supposed to download the archive and unpack it to $SPLUNK_HOME/etc/apps

Restart Splunk

That's it. No ingesting anything, no defining inputs, no nothing. The files will _not_ be moved anywhere - the app contains pre-indexed buckets along with the indexes.conf file pointing to this particular directory so that Splunk knows where to find the data. So after the restart Splunk should notice that it has new index(es?) with data files placed in your app's directory (that's kinda unusual and you'd normally not do that for normally ingested index data but that's a dataset prepared to be easily distributed). And that's all there is to it.

You should _not_ be ingesting it in any way which you somehow did since you're showing us the contents of the files pulled into some index.

Re: I cant get Splunk to read Boss Of SOC v1 Data Set

dablab — Sat, 18 Nov 2023 18:20:01 GMT

Thanks!

So when I do

cd /opt/splunk/etc/apps/

and then: sudo tar -xzf botsv1_data_set.tgz

It will unload the data in the apps area, but then when I restart and go to search it there is nothing there?

I have all the apps download etc.

Re: I cant get Splunk to read Boss Of SOC v1 Data Set

dablab — Sat, 18 Nov 2023 18:48:20 GMT

I want to say that it was a permissions issue!!! Thanks all!

Re: I cant get Splunk to read Boss Of SOC v1 Data Set

PickleRick — Sat, 18 Nov 2023 19:58:46 GMT

Was just about to write that if you unpacked it with sudo, you could get mismatched ownership and permission issues. But apparently you got it on your own.

Have fun with your searches 🙂