charset issue

[perlish]

Hi, everybody.

I want use splunk to index the data which contain chinese.

Firstly, the base data will send to my splunk universal forwarder.

Then,my universal forwarder will forward the data to my splunk.

The base data char is gb2312.

This is the inputs.conf in universal forwarder.

[udp://514]
connection_host = none
sourcetype = businesslog

This is the props.conf in splunk.

[businesslog]
CHARSET = GB2312

But now the chinese still can`t display correctly.

Wheather i need to define the props.conf in universal forwarder?

Thanks.

[acharlieh]

CHARSET is an attribute set into the pipeline data at input time, the props.conf should be on your Universal Forwarders. References:

• http://answers.splunk.com/answers/132918/charset-does-not-effect-when-data-is-from-a-universalforwar...
• http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F
• http://wiki.splunk.com/Community:HowIndexingWorks

[woodcock]

This props.conf setting is an index-time setting so it needs to be certain places depending on your configuration. If you are using Heavy Forwarders, it must be on your forwarders but if you are using Universal or Light-Weight Forwarders, it must be on (all of) your Indexers; is it on all of your Indexers?

[acharlieh]

It's checked at parse time, however it's an input time setting. Otherwise the UF would not be able to know if it was cutting multibyte characters in half or not as it sends chunks of data on to the parser.

[woodcock]

Fair enough but my point stands that this props.conf should be exported to every node that needs to modify the data (forwarders and indexers) and my suspicion is that it has not been so distributed.