Getting Data In

Why do special characters "[0[0m" appear in my events?

pmerlin1
Path Finder

Hi

I deploy Splunk forwarder on a JBoss server to forward data towards my test environment Splunk.

In the Universal Forwarder (UF) monitor file server.log file, the line

01/12/16 15:11:50,398 INFO  [org.jboss.as] (MSC service thread 1-3) JBAS015950: JBoss EAP 6.4.8.GA (AS 7.5.8.Final-redhat-2) stopped in 358ms

is transfomed by the event below

01/12/16 15:11:50,398 INFO  [org.jboss.as] (MSC service thread 1-3) JBAS015950: JBoss EAP 6.4.8.GA (AS 7.5.8.Final-redhat-2) stopped in 358ms

All lines is prepended by characters [0m[0m for INFO message or [0[31m when it's a ERROR message

Someone can explain why?

0 Karma
1 Solution

pmerlin1
Path Finder

Thanks gokadroit.
I solve my color syntaxing issue, in modfying my custom java log handler . I dont want report the clean effort on Splunk. I remove %K{level}% of the pattern in the logger configuration.
Now it works the event is display without color codes.

View solution in original post

0 Karma

pmerlin1
Path Finder

Thanks gokadroit.
I solve my color syntaxing issue, in modfying my custom java log handler . I dont want report the clean effort on Splunk. I remove %K{level}% of the pattern in the logger configuration.
Now it works the event is display without color codes.

0 Karma

gokadroid
Motivator

A shot in the dark will be the log lines might have colour coding info in them to appear colourful on console, however when they are sent over to Splunk this colour coding stays preserved as part of log line. Generally the colour code numbers are as follows clubbed together with [ and m:

Black        0;30     Dark Gray     1;30
Red          0;31     Light Red     1;31
Green        0;32     Light Green   1;32
Brown/Orange 0;33     Yellow        1;33
Blue         0;34     Light Blue    1;34
Purple       0;35     Light Purple  1;35
Cyan         0;36     Light Cyan    1;36
Light Gray   0;37     White         1;37

You can try to rex them out using something like this:

your query to return the event
| rex mode=sed field=_raw "s/\[[0-9]+m//g"
| complete your query
0 Karma

mahendra5
New Member

This solution is working for us very well.

I can combine it with my query and rex does the job. I can create macros too.

| rex mode=sed field=_raw "s/[[0-9]+m//g"

0 Karma

pmerlin1
Path Finder

Thank you gokrakoid
I prefer to strip the color codes before indexing
I try in transforms.conf on my indexer :
[MyLOG-stripper-colorcodes]
REGEX = ^[\d+m[\d+m(.*)$
FORMAT = $1
DEST_KEY = _raw

but it deosn't work. The characters are not removed still. Any idea ?

0 Karma

gokadroid
Motivator

Can you try to escape any special characters like [ to make it \[ or try to see if sedcmd can assist to do that in props.conf?

[your:sourcetype]
  SEDCMD-rep_1 = s/\[[0-9]+m//g

Please note that you will have other properties as well in props.conf so do not remove them for your:sourcetype. Just append the line SEDCMD-rep_1 = s/\[[0-9]+m//g to existing properties.

0 Karma

mahendra5
New Member

@gokadroid
this works perfect, sourectype can easily eliminate those before even it gets indexed.

0 Karma

ddrillic
Ultra Champion

Interesting, we see these characters here as well - archive.travis-ci.org

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...