Getting Data In
Highlighted

Why do special characters "[0[0m" appear in my events?

Path Finder

Hi

I deploy Splunk forwarder on a JBoss server to forward data towards my test environment Splunk.

In the Universal Forwarder (UF) monitor file server.log file, the line

01/12/16 15:11:50,398 INFO  [org.jboss.as] (MSC service thread 1-3) JBAS015950: JBoss EAP 6.4.8.GA (AS 7.5.8.Final-redhat-2) stopped in 358ms

is transfomed by the event below

01/12/16 15:11:50,398 INFO  [org.jboss.as] (MSC service thread 1-3) JBAS015950: JBoss EAP 6.4.8.GA (AS 7.5.8.Final-redhat-2) stopped in 358ms

All lines is prepended by characters [0m[0m for INFO message or [0[31m when it's a ERROR message

Someone can explain why?

0 Karma
Highlighted

Re: Why do special characters "[0[0m" appear in my events?

Ultra Champion

Interesting, we see these characters here as well - archive.travis-ci.org

0 Karma
Highlighted

Re: Why do special characters "[0[0m" appear in my events?

Motivator

A shot in the dark will be the log lines might have colour coding info in them to appear colourful on console, however when they are sent over to Splunk this colour coding stays preserved as part of log line. Generally the colour code numbers are as follows clubbed together with [ and m:

Black        0;30     Dark Gray     1;30
Red          0;31     Light Red     1;31
Green        0;32     Light Green   1;32
Brown/Orange 0;33     Yellow        1;33
Blue         0;34     Light Blue    1;34
Purple       0;35     Light Purple  1;35
Cyan         0;36     Light Cyan    1;36
Light Gray   0;37     White         1;37

You can try to rex them out using something like this:

your query to return the event
| rex mode=sed field=_raw "s/\[[0-9]+m//g"
| complete your query
0 Karma
Highlighted

Re: Why do special characters "[0[0m" appear in my events?

Path Finder

Thank you gokrakoid
I prefer to strip the color codes before indexing
I try in transforms.conf on my indexer :
[MyLOG-stripper-colorcodes]
REGEX = ^[\d+m[\d+m(.*)$
FORMAT = $1
DEST_KEY = _raw

but it deosn't work. The characters are not removed still. Any idea ?

0 Karma
Highlighted

Re: Why do special characters "[0[0m" appear in my events?

Motivator

Can you try to escape any special characters like [ to make it \[ or try to see if sedcmd can assist to do that in props.conf?

[your:sourcetype]
  SEDCMD-rep_1 = s/\[[0-9]+m//g

Please note that you will have other properties as well in props.conf so do not remove them for your:sourcetype. Just append the line SEDCMD-rep_1 = s/\[[0-9]+m//g to existing properties.

0 Karma
Highlighted

Re: Why do special characters "[0[0m" appear in my events?

New Member

@gokadroid
this works perfect, sourectype can easily eliminate those before even it gets indexed.

0 Karma
Highlighted

Re: Why do special characters "[0[0m" appear in my events?

New Member

This solution is working for us very well.

I can combine it with my query and rex does the job. I can create macros too.

| rex mode=sed field=_raw "s/[[0-9]+m//g"

0 Karma
Highlighted

Re: Why do special characters "[0[0m" appear in my events?

Path Finder

Thanks gokadroit.
I solve my color syntaxing issue, in modfying my custom java log handler . I dont want report the clean effort on Splunk. I remove %K{level}% of the pattern in the logger configuration.
Now it works the event is display without color codes.

View solution in original post

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.