topic Re: What is the best way to spoof run-anywhere fake data for a question? in Splunk Search

What is the best way to spoof run-anywhere fake data for a question?

woodcock — Tue, 07 Jul 2015 04:27:05 GMT

Many people ask questions here that are tricky enough that the only way to get an answer that works is to play around with the data quite a bit. In order to do this, we have to fake data first. For the following data set, what is the best way to do it?

host   source  count name
host1  sourceA 33    Inky
host2  sourceA 23    Pinky
host3  sourceB -2    Blinky
host4           5    Clyde

What about for multi-value fields?

Re: What is the best way to spoof run-anywhere fake data for a question?

esix_splunk — Tue, 07 Jul 2015 04:33:35 GMT

A combination of _internal, eventgen, and the Windows or NIX TA.

These can cover most all general questions and functionality of Splunk Search related questions. There almost examples of almost all kinds of datasets included in these, and pouring over these is a great way to learn Splunk.

Re: What is the best way to spoof run-anywhere fake data for a question?

acharlieh — Tue, 07 Jul 2015 05:05:50 GMT

"Best" is rather subjective, and varies widely with the question being asked. However I try to use the most straightforward method as needed for a particular problem. In your example case As you gave sample data in a tabular form I would use |noop|stats count to get a single result record, followed by eval to paste in your table as _raw then use multikv to split into records and fields. As I previously pointed out in a comment.

Multivalve fields would handle no differently, just pick a delimiter that doesn't appear elsewhere and serialize multivalued fields with it and after multikv, use eval with the split function to make your mv field. (I used a similar principle in the original part of this answer encoding multivalued fields as single valued fields with *** as a delimiter)

When data isn't provided using gentimes can generate a bunch of time slots quickly. A few evals making use of random() with appropriate math to constrain ranges and you have magically generated massive sets of data. Using summary indexing commands and temporary indexes then let's you keep that generated set and see what manipulation scan be done.

Another method I've used was to take a string, split it into a multivalued field, mvexpand and then used auto extraction. in this gist. I don't remember if I generated or was given that data.

But in short, there is no best way, use the many tools of Splunk and do whatever is easiest for the problem at hand.

Re: What is the best way to spoof run-anywhere fake data for a question?

martin_mueller — Tue, 07 Jul 2015 05:16:27 GMT

I frequently do something like this:

| stats count | eval field = "val1 val2 val3" | makemv field | mvexpand field
| eval mv = "mv1 mv2 mv3" | makemv mv
| streamstats count | eval val = random()%100
| eval _time = now() + random()%100 | sort - _time

Gives you five things:

three events to play with
single- and multi-value fields
a count or id
numerical data
timestamps

Pick what you need and re-assemble for each sample data task.

Re: What is the best way to spoof run-anywhere fake data for a question?

woodcock — Tue, 07 Jul 2015 05:29:45 GMT

I typically do something very similar to @martin_mueller like this:

| metadata type=hosts | head 1 | eval name="Inky Pinky Blinky Clyde" makemv name | mvexpand name

This gives me my 4 events and it does it quickly because nothing is faster than "head 1" (I think).
Now I can set my other fields' values with case statements like this:

| eval host=case(name="Inky", host1, name="Pinky", host2, name="Blinky", host3, name="Clyde", host4)

Re: What is the best way to spoof run-anywhere fake data for a question?

martin_mueller — Tue, 07 Jul 2015 06:50:11 GMT

Technically, this is much faster than | head 1:

| noop | stats count | ...

Can't get faster than not even loading one event...

Re: What is the best way to spoof run-anywhere fake data for a question?

woodcock — Fri, 02 Oct 2015 13:37:24 GMT

Accept that you don't need the noop part:

| stats count | ....

Re: What is the best way to spoof run-anywhere fake data for a question?

acharlieh — Fri, 02 Oct 2015 14:02:27 GMT

Need? Maybe not, but talking in terms of pure speed.... It's been a while since I did this experiment, but use the job inspector and compare performance metrics of | stats count and |noop | stats count if I remember correctly, in a distributed environment the former is actually distributing a search (that happens to return nothing from anywhere), but the latter literally does nothing and counts it. So the former is impacted by connectivity to indexers, and the latter is not. 🙂

Re: What is the best way to spoof run-anywhere fake data for a question?

woodcock — Fri, 02 Oct 2015 15:39:42 GMT

Brilliant; I would not have even thought to check this!

Re: What is the best way to spoof run-anywhere fake data for a question?

martin_mueller — Sun, 13 Dec 2015 22:48:28 GMT

Small update, since 6.3 there is a dedicated command to make artificial results: http://docs.splunk.com/Documentation/Splunk/6.3.1/SearchReference/Makeresults

Re: What is the best way to spoof run-anywhere fake data for a question?

woodcock — Wed, 29 Mar 2017 19:14:15 GMT

There is a new command for this that can be used instead of |noop|stats count AS ..., it is |makeresults:

https://docs.splunk.com/Documentation/Splunk/6.5.2/SearchReference/Makeresults

Re: What is the best way to spoof run-anywhere fake data for a question?

woodcock — Wed, 05 Apr 2023 21:08:26 GMT

The best way is like this:

| makeresults | eval _raw=" host source count name host1 sourceA 33 Inky host2 sourceA 23 Pinky host3 sourceB -2 Blinky host4 5 Clyde" | multikv forceheader=1

Re: What is the best way to spoof run-anywhere fake data for a question?

PickleRick — Wed, 05 Apr 2023 21:23:07 GMT

It is a nice way but it works great only if you're copy-pasting from somewhere. If you're producing data by hand this way, you need all this spaces counting. I'd rather do a simple json and then spath/mvexpand if needed. But I usually generate data with something like for example

| makeresults count=100
| streamstats count
| eval fieldA=count % 12
| eval fieldB=count % 7
| [... and so on ...]