Getting Data In

How can I get datasets/logs for monitoring and analysis?

kattey
New Member

Hello, good day
I am very new to Splunk, i and my team want to work on a mini project using splunk cloud with the topic "Splunk Enterprise: An organization's go-to in detecting cyberthreats"
how/where can i easily get datasets/logs that i can use in splunk for monitoring and analysis.  and what best way should we go about this topic?

Labels (1)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @kattey ,

how much do you know Splunk?

if you start from scratch you need to learn hot to ingest data in Splunk and how to search on Splunk.

Data sources com from you infrastructure, if you haven't, you could use an automatic generator, but it isn't another stack to learn!

Best way, is to search in Community answers about basic learning (e.g. Search Tutorial) and getting data in.

Then you should define a perimeter to identify the data sources to ingest.

Ciao.

Giuseppe

inventsekar
SplunkTrust
SplunkTrust

Hi @kattey ... please check these things:

1) As i heard, the Splunk Essentials app got some sample data. 

https://splunkbase.splunk.com/app/3435

2) and then you can find some sample data in this repo:

https://github.com/splunk/botsv3

3) and then, there is an app.. EventGen. very difficult to configure and very worst documentation. i would suggest this as last resort. thanks. 

4) Splunk Datasets Add-On: This Splunk add-on provides a variety of sample data sets, including security logs, for you to work with. You can download and install the add-on directly from Splunkbase: https://splunkbase.splunk.com/app/3245/

5) Boss of the SOC (BOTS) datasets: You've already mentioned BOTS v1-3, but don't forget about BOTS v4, which was released later. You can find it here: https://github.com/splunk/botsv4

6) Elastic Common Data Model (ECS) sample data: Although intended for the Elastic Stack, you can adapt these sample logs for use in Splunk. The repository contains logs from various sources, such as network traffic, security events, and web server logs: https://github.com/elastic/ecs/tree/master/generated/samples

6) Sample Log Generator: This tool generates synthetic logs that you can customize to fit your needs. While not real-world data, it can be useful for testing specific scenarios or queries: https://github.com/ErikEJ/SqlQueryStress

7) NIST National Vulnerability Database (NVD) data feeds: NVD provides various data feeds containing vulnerability information. While not logs per se, this data can be useful for exploring security-related data in Splunk: https://nvd.nist.gov/vuln/data-feeds

😎 SecRepo: You've already mentioned this repository, but I'd like to emphasize its value as it contains various sample logs from different sources: http://www.secrepo.com/ 

9) https://github.com/gfek/Real-CyberSecurity-Datasets

10) https://github.com/shramos/Awesome-Cybersecurity-Datasets

11) https://www.secrepo.com/

 

hope this helps you and other splunkers.. thanks. karma / upvotes appreciated by all, thanks. 

thanks and best regards,
Sekar

PS - If this or any post helped you in any way, pls consider upvoting, thanks for reading !
Get Updates on the Splunk Community!

SignalFlow: What? Why? How?

What is SignalFlow? Splunk Observability Cloud’s analytics engine, SignalFlow, opens up a world of in-depth ...

Federated Search for Amazon S3 | Key Use Cases to Streamline Compliance Workflows

Modern business operations are supported by data compliance. As regulations evolve, organizations must ...

New Dates, New City: Save the Date for .conf25!

Wake up, babe! New .conf25 dates AND location just dropped!! That's right, this year, .conf25 is taking place ...