log with key value pair or transforms.conf performance diffrence?

[jazzythemartian]

Hi,

to gain index size I made the log format as below. I didn't use key value pair.

20121101095842|192.168.1.2|KRQQQShcnQdRK8pLKTXC|20138494756382|I|PLAY|this the detailed info|1

And in transforms.conf I defined the fields.
DELIMS="|"
FIELDS=time,sourceip,session_id,customer_id,channel,op_type,detail,result_code

What if I made the log format like;

time=20121101095842,sourceip=192.168.1.2,sessiın_id=KRQQQShcnQdRK8pLKTXC,customer_id=20138494756382,channel=I, op_type=PLAY, detail=this the detailed info|result_code=1

Is there any performance diffrence between these two? a big diffrence in speed?

thanks,

a.

[kristian_kolb]

Well, as you've probably calculated, you'll save some license space - in this case like 40%. I cannot see any immediate downside to the approach - as long as you keep the number and order of fields constant. With key=value pairs, that is not relevant, as the extraction takes place automatically.

You should probably set KV_MODE=none for this sourcetype in props.conf.

http://docs.splunk.com/Documentation/Splunk/latest/Admin/Propsconf

Whether a REPORT is faster than KV_MODE=auto... I don't know - perhaps a little.

/K

[sowings]

I agree with your gut.

[kristian_kolb]

Naturally - having both is the worst 🙂

Gut feeling says that REPORT + KV_MODE=none should be faster than KV_MODE=auto. Should be fewer, less complicated steps. Though for some searches the difference might not be even noticeable.

[sowings]

REPORT with DELIMS is definitely faster if you turn off KV_MODE=auto for that type. 🙂 I'm not sure if "properly configured" REPORT with DELIMS alone is faster than key=value pairs, however.