Eventgen: is it possible to create events taking pair values from a file?

[gcusello]

Hi at all,

I have to use eventgen to populate a demo I prepared.

I'm able to populate events starting from a template and changing some values in a token from a file.

My question is: is it possible to populate two tokens correlated between them?

in other words, in my events I have two tokens:

• token1
• token2

if in my file I have the following table:

value1 message1
value2 message2
value3 message3

is it possible to put in the same events?

• if value1 is in token1 -> message1 is in token2,
• if value2 is in token1 -> message2 is in token2,
• if value3 is in token1 -> message3 is in token2.

To do this can I put vales and messages in the same file or I have to put them in two different files?

in in two different files, how can I be sure that they are always in the correct position?

Ciao.

Giuseppe

[bowesmana · Accepted solution]

@gcusello 

I started off writing that it's not possible, but then I re-read your question again and this is how it's done. For example, my au_identities.csv contains a list of users, with all data for the same user on the same row, so this will set the tokens to the corresponding 1,4,5th fields from that same row.

token.0.token = ##date##
token.0.replacementType = timestamp
token.0.replacement = %Y-%m-%d %H:%M:%S

token.1.token = ##userid##
token.1.replacementType = mvfile
token.1.replacement = $SPLUNK_HOME/etc/apps/fraudgen/lookups/au_identities.csv:1

token.2.token = ##first##
token.2.replacementType = mvfile
token.2.replacement = $SPLUNK_HOME/etc/apps/fraudgen/lookups/au_identities.csv:4

token.3.token = ##last##
token.3.replacementType = mvfile
token.3.replacement = $SPLUNK_HOME/etc/apps/fraudgen/lookups/au_identities.csv:5

However what I originally wrote...

There are two alternatives

Build your own set of saved searches that define logic to create your sample data using lookups and collect statements to build your data set. I have used this quite a lot to generate demo data sets that are simply not possible with eventgen. For example, it's not possible with eventgen to reference a particular 'ID' that has been generated in a previous event so that you can add 'transaction' type data for that ID.

I have found through creating numerous demo data sets that the saved search approach gives me all the flexibility needed with the full power of SPL to do pretty much anything. So, for your example it's a simple randomised lookup followed by an eval/case statement to set the 

You still use the same lookup files you would use for eventgen but you can then start to create much richer data sets with your own weighting mechanisms built into your SPL. _time can be set to whatever, based on your own logic. It's probably more complex to do it this way, but you get way more control.

Another option is to look at the 'other' event generation tool - but I cannot remember its name - one of the Splunk guys referred to it a last years Partner Tech Symposium, but it does allow more complex transaction type relationships.