topic Re: How many field extract in splunk? in Monitoring Splunk

How many field extract in splunk?

cucuro — Fri, 05 Apr 2013 06:17:56 GMT

Hello Everyone,

Now a days when I extract fields, need 1000ea fields.
so, i defined in transforms.conf and props.conf about 1000 ea fields.
But Splunk can not recognizes.

I configured.

props.conf

[sourcetype]
REPORT-sourcetype = delimExtractions_CC_type

transforms.conf

[delimExtractions_CC_type]
DELIMS = "|"
FIELDS = "time", "A0001","A0002","A0003","A0004",.... "A1713","A1714","A1715"

I need add something in transforms.conf and props.conf?

Re: How many field extract in splunk?

kristian_kolb — Fri, 05 Apr 2013 07:14:07 GMT

I assume that in props.conf, it does not literally say 'sourcetype', but rather the actual sourcetype of your data.

Also, are your events really that long? 1500+ pipe-separated fields? Always the same length?

Please post a sample event.

Re: How many field extract in splunk?

cucuro — Fri, 05 Apr 2013 07:32:11 GMT

2012-09-06 02:59:59.0|0|0|0|0|0|0|0|1|0|0|0|0|0|0|1|0|0|0|1|0|0|1|0|0|0|0|1|0|1|0|0|0|0|0|0|0|0|0|0|1|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|725|0|25903|184|181|18|364|137|674|2720|12404|12427|12419|17844|7.6|2.8|24.4|4.4|122.0|53.2|374.7|55.3|65.9|79.3|83.8|81.9|83.9|84.9|85.5|82.6|83.7|84.2|81.4|46.3|46.5|43.1|45.4|43.8|47.1|40.4|46.8|464.7|41.7|81.7|82.2|80.5|81.8|70.5|68.6|0.93|60.34|..............

Re: How many field extract in splunk?

cucuro — Fri, 05 Apr 2013 07:34:43 GMT

some 0 or 1, some integer, some flot etc... fields 1872 ea.

So many fields...

Re: How many field extract in splunk?

eashwar — Fri, 05 Apr 2013 11:17:25 GMT

wow man!!! i am really waiting to see the answer for this question!!! 1500+ fields how do you analyse ?

Re: How many field extract in splunk?

cucuro — Fri, 05 Apr 2013 11:21:43 GMT

so i need help. ^^;

Re: How many field extract in splunk?

lguinn2 — Sun, 07 Apr 2013 01:08:34 GMT

You could try adding the following to $SPLUNK_HOME/etc/system/local/limits.conf

[kv]
maxcols = 1750

By default, Splunk does not extract more than 512 fields, for performance reasons.

You might also check to make sure that your events are not being truncated, although it looks like they are okay.