topic Re: Integrating a series of flat values into Splunk in Getting Data In

Integrating a series of flat values into Splunk

srw46 — Tue, 29 Jun 2010 22:35:34 GMT

Hello all,

I'm on the fish for ideas or anybody who has previous experience with this.

Essentially, we have two tables of (mostly) fixed data which we would like to 'teach' Splunk (for want of a better term).

To put it in context we have throughput files that report a transaction ID and a transaction time, Splunk grabs these fields no problem. Elsewhere in some flat tables we have transaction names (that relate to an ID) and a time threshold for each transaction time.

Is there anyway we can bring this data into the mix? If Splunk can know about the average for each transactions, and compare to the actual times (our main concern) and if it could line up the arbitrary transactions ID's with the meaningful names it would make analysis of the logs inifnitely more useful.

I'm a bit of a Splunk noob (actually, a lot of one) so sorry if there is precedent for this or some glaringly obvious answer. Really just looking for any sort of starting point.

Thanks in advance for any advice you can give. I can elaborate further if need be.

Re: Integrating a series of flat values into Splunk

Lowell — Tue, 29 Jun 2010 23:50:00 GMT

I'm not "100%" sure what you mean by flat tables, but it sounds like what you are looking for is splunk's lookup mechanism, which is new in Splunk 4.x.

You have two different options for lookups:

Simple flat file (*.csv)
Scripted lookups (you write a small python script which does the heavy-lifting; which lets you do whatever kind of lookup you'd need, like a SQL query, internet lookup, or whatever else you need.)

Docs:

Look up fields from external data sources

Re: Integrating a series of flat values into Splunk

srw46 — Mon, 05 Jul 2010 16:03:53 GMT

Thank you Lowell, this is indeed what we were looking for!

Re: Integrating a series of flat values into Splunk

Lowell — Tue, 06 Jul 2010 01:46:52 GMT

Great. If this answers your question, you can indicate so by click the check mark on the side.