All Apps and Add-ons

How to Get Eventgen working?

skoelpin
SplunkTrust
SplunkTrust

I installed eventgen and watched the tutorial videos. I created a new app, changed the permissions to global, created a sample folder inside the new app, dropped my sample file in /opt/splunk/etc/apps/internal_app/samples, then moved the eventgen.conf.tutorial file to /opt/splunk/etc/apps/internal_app/local, renamed it to eventgen.conf, uncommented out the lines, referenced my sample file in the stanza, and restarted Splunk. After restarting, I see zero data flowing into the main index which is the one I specified in eventgen.conf.

When I do a search on the internal logs, /opt/splunk/var/log/splunk/eventgen.log I can see my data sample along with Splunk saying it backfilled successfully.

I'm stumped as to why this isn't generating data

Here's my eventgen.conf file

[Test_Data.txt]
mode = replay
sampletype = csv
timeMultiple = 2
backfill = -15m
backfillSearch = index=main sourcetype=eventgen

outputMode = stdout
outputMode = splunkstream
splunkHost = localhost
splunkUser = admin
splunkPass = changeme


token.0.token = \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3,6}
token.0.replacementType = timestamp
token.0.replacement = %Y-%m-%d %H:%M:%S,%f

Here's what I'm seeing in the internal logs

2017-12-21 14:01:38,605 INFO module='Timer' sample='Test_Data.txt': Stopping timer for sample 'Test_Data.txt'
host =  MCxxxxxxxxx source =/opt/splunk/var/log/splunk/eventgen.log sourcetype =eventgen

12/21/17 1:54:11.032 PM 
2017-12-21 13:54:11,032 INFO module='Timer' sample='Test_Data.txt': Backfill complete

inventsekar
Super Champion

EventGen is an important tool and at the same time, making it work looks like a very difficult task.
Poor documentation. God please save me.
Adding to the issue, like adding fuel to the fire, that video mentioned was removed.

PS ... If any post helped you in any way, pls give a hi-five to the author with an upvote. if your issue got resolved, please accept the reply as solution.. thanks.

cboillot
Contributor

I, too, am not able to get this to work at all.

0 Karma

Ohiotech
Explorer

After installing the event gen manually (did not path correctly with directions), I added the SA-Eventgen app through the manage apps\ install from file. The trick for me was to go into the Data Input/SA-Eventgen input and enable it. (disabled during install). Events were there within minutes after a restart.

ian_thomas
Path Finder

This fixed my issue. Thanks!

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Hi @skoelpin,

I have created sample app with eventgen configuration and it is working fine.

Steps which I have followed
1. Installed eventgen app on splunk.
2. Created new app with name "test_app"
3. Created $SPLUNK_HOME/etc/app/test_app/default/eventgen.conf with below content

[test_data\.txt]
interval = 60
earliest = -60m
latest = now
sourcetype = test_st
source = eventgen
disabled=0
token.0.token = TTTTTTTT
token.0.replacementType = timestamp
token.0.replacement = %Y-%m-%d %H:%M:%S,%f
  1. Created $SPLUNK_HOME/etc/apps/test_app/samples/test_data.txt file with below content

    TTTTTTTT transType=ReplaceMe transID=000000 transGUID=0A0B0C userName=bob city="City" state=State zip=00000 value=0
    
  2. Below content added into $SPLUNK_HOME/etc/apps/test_app/metadata/default.data

       [eventgen]
    access = read : [ * ], write : [ admin ]
    export = system
    
  3. Restarted splunk service
    And I can see data in splunk with query index=main sourcetype=test_st and events generated with eventgen are

       2017-12-22 16:19:59,595283 transType=ReplaceMe transID=000000 transGUID=0A0B0C userName=bob city="City" state=State zip=00000 value=0
    2017-12-22 16:17:11,647706 transType=ReplaceMe transID=000000 transGUID=0A0B0C userName=bob city="City" state=State zip=00000 value=0
    

    EDIT:

Tried with eventgen.conf which is provided in original question with some modification

[test_data\.txt]
mode = replay
timeMultiple = 2
backfill = -60m
backfillSearch = index=main source=eventgen

outputMode = splunkstream
splunkHost = localhost
splunkUser = admin
splunkPass = changeme

token.0.token = \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3,6}
token.0.replacementType = timestamp
token.0.replacement = %Y-%m-%d %H:%M:%S,%f

And $SPLUNK_HOME/etc/apps/test_app/samples/test_data.txt with content

2017-10-14 11:12:13,567 transType=ReplaceMe transID=000000 transGUID=0A0B0C userName=bob city="City" state=State zip=00000 value=0

This is also backfilling data perfectly fine.

It looks you don't have any data in splunk while running backfill query index=main sourcetype=eventgen that's why eventgen is not backfilling any data.

I hope this helps.

Thanks,
Harshil

0 Karma

skoelpin
SplunkTrust
SplunkTrust

Thanks for your detailed response. I copied your directions and have an identical setup to yours and it is still not streaming data in with that sourcetype. Does it take awhile before it starts generating?

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

It will generate events within 5 minutes after splunk restart. Can you please post some logs from eventgen.log file for test_data.txt file.

0 Karma

Penkov
Loves-to-Learn

Hi, harsmarvania57,
Thanks for your detailed response.But i have the same problem like "skoelpin" and i wait maybe 10 or 15 minutes and i don't receive any events from eventgen.

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Hi @Penkov,

I am not sure what problem are you facing but you can try Gogen instead of Eventgen https://github.com/coccyx/gogen , this also generate dummy data but I never tried this.

0 Karma

Penkov
Loves-to-Learn

Thanks @harsmarvania57 for your answer. My problem is that i can't run eventgen and create any type of event. I try to copied your directions and have an identical setup to yours and it is still not streaming data in with that sourcetype.

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Which version of Splunk are you running ? and Is it standalone instance, if not standalone then on which instance you installed eventgen ? It will be good to start new question with required information and refer this question link in your new question.

0 Karma

naidusadanala
Communicator

Replace [Test_Data.txt] with [Test_Data] , Restart the server.

It should work.

Please let us know

0 Karma

mayurr98
Super Champion

Hey

Use this link

https://youtu.be/9S-ZeGEfRKg

1) download eventgen from https://github.com/splunk/eventgen
2) extract it to etc/apps/SA-Eventgen (yes, it should be named that to eventgen).
3) review Eventgen in Manage Apps, you may want to make it visible.
4) Restart Splunk
eventgen.conf and samples in them that are ready to go

0 Karma

skoelpin
SplunkTrust
SplunkTrust

I've watched that video about 4x over now and have an identical setup and it will not generate events!

0 Karma
Get Updates on the Splunk Community!

Introducing Ingest Actions: Filter, Mask, Route, Repeat

WATCH NOW Ingest Actions (IA) is the best new way to easily filter, mask and route your data in Splunk® ...

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...