Splunk Search

Splunk not displaying log data

Builder

My log file contain a long line (35000 chars) with continuous spaces [more then 60 spaces] multiple times inside the log - I can't see this log information in Splunk.

I can't change the log because its coming from 3rd party tool

Any Idea?

Tags (2)
1 Solution

Ultra Champion

Have you looked at the TRUNCATE parameter for props.conf?

By default Splunk will cut lines that are longer than 10000 chars. Changing this to 50000 might improve your situation.

Other things to check - if you are not seeing the data at all - are the simple things;

Look for interesting error messages in splunkd.log
Are timestamps parsed correctly? Try to search for 'All Time'
Are you looking in the right index?
Do you have permissions to read that index? Check in the Manager -> Account controls -> roles -> your role

Hope this helps,

Kristian

View solution in original post

Ultra Champion

Have you looked at the TRUNCATE parameter for props.conf?

By default Splunk will cut lines that are longer than 10000 chars. Changing this to 50000 might improve your situation.

Other things to check - if you are not seeing the data at all - are the simple things;

Look for interesting error messages in splunkd.log
Are timestamps parsed correctly? Try to search for 'All Time'
Are you looking in the right index?
Do you have permissions to read that index? Check in the Manager -> Account controls -> roles -> your role

Hope this helps,

Kristian

View solution in original post

Builder

I can't see this error anymore

0 Karma

Builder

Hi Drainy, can you please tell me how to remove TAB and space char before indexing?

0 Karma

Champion

You can remove them before indexing, but its unlikely that Splunk is struggling with them. It could be the Web UI struggling to render them perhaps but I am dubious that it isn't dealing with the tab or spaces as at a very basic level they are just another character.

0 Karma

Builder

Thanks for your quick response.
in splunk is there any option to remove tab char or spaces between > < char before forwarding?

0 Karma

Champion

Sorry, I've been out and abooot. Well Splunk shouldn't have any issue with that really, if anything it could be a line-breaking issue with how Splunk is splitting the data. Perhaps you could paste it to pastebin.com, then we can grab a decent copy to test on our own systems

0 Karma

Builder

Hi Drainy any update regarding my query?

0 Karma

Builder

I0705 13:59:35.496698 19354 test.cpp:500] test::call: setParam(i=0, 'TEST.XML,,,,,,<<?xml version="1.0" encoding="UTF-8"?>###.APPLICATION162588014750291.03:5 tab1 US0010001 ###.APPLICATION #####.######## , I VALIDATE ID003350000000F3 ########FTI_1341493112469 <####S##een>#########.##############_801475029102</####S##een> fieldName:###########.#### fieldName:#####.####.## fieldName:#####.######## fieldName:#####.###### fieldName:######.####.## <###ordRead>1</###ordRead> ##.###.###.## #########.#### ####.####.#####.### ####.####.##### 1 ####.####.##.#### ####.####.## 1 ############## ############### ############## ############### ############# ########## ############ ########## ############# ################# ############### ########### ###.##### 1 ###.######## 1 ###.###.####### ###########.## ##.#########.#### ##.##.##.#### 1 ##.##.##.### 1 1 ####.###### ####.###.###### 1 ######.### 1 ######.#### ######.COM.DISPLAY ######.### 1 ######.#### 1 ########.######## ######.####.## ####.#### ######.##### ######.###### ###.######.### ###.###.###.### ##.#### COLL.###.## ##########.### 1 ##########.#### ##########.### 1 ##########.#### 1 #####.###### ##.######.###D.Y.N ######.####.## IBMUSD ######.###### ######.####.#### ######.######## ######.######## ######.#####.### ######.#####.#### ####.## ########.###.## 1 ########.###.DR 1 ####.GROUP.LEVEL ########.#### ########.SPREAD DATE.TIME 1 DEAL.MARKET DEALER.DESK #####.####.## DELLUSD #####.###### 1000 #####.####.#### #####.######## USD #####.######## #####.#####.### #####.#####.#### ########.##### ########.### ########.###### 1 ####.#### DR.######.###D.Y.N DRAWN.###OUNT EXPECTED.#####.## EXPECTED.###S.## EXPOSURE.#### EXTEND.###MAT EXTEND.#### FED.##### FREE.TEXT.###TO 1 ##.###.#####.### ##.###.#####.## 1 ##.####.#####.### ##.###.####.## ##.###.##### 1 ##.###.######## 1 ##.###.###.####### ##.##.##.## 1 ##.#.###.###.## 1 ##.#.##.#.##.## 1 ##.#.##TMED.## 1 ##.#.###.## 1 ##.#####.####.## ##.#####.VDATE ##.EXCH.#### ##.##STR.#### 1 ##.##TERMED.### ##.##TERMED.## 1 ##.##TMED.##### 1 ##.###.####.#### ##.ORDERING.## 1 ##.ORDERING.CUS 1 ##.###.######S 1 ##.PROCESS.ERR 1 ##.REASON.OVE 1 ##.###.### ##.###.####.### ##.###.####.## 1 ##.####.### 1 ##.####.####.## 1 ##.#####.### 1 ##.TIME.### 1 INP.###.### ######## 1 INSTRCTN.#### 1 ##########.### #######.#####.### #######.##### 1 ###.####.### ######.###.#### ######.######.#### ###.###.######## ###.###.######## ###.####.### ###.###.###S.### ###.##T.###.### #####.#.######.### #####.#.##### MAILING MESSAGE.#### 1 MESSAGE 1 ####.####.###### ###.#### 1 ###.#### 1 #####.#### ###.######.###NO #######.###### ##.##.#####.##### ###.####.#### ###.####.#### ORDERING.##### 1 ORDERING.#### 1 OVERRIDE 1 ######.###.## ###.###### #####.#### #######.######S 1 ########.#### ##########.### ##########.#### ######.######.#### ######.######.#### ####.###### ####.######.### ####.#####.### ###.####.#####.### ###.####.##### 1 ######.###.#### ########.##### ######.###### ########.### 1 ########.### ########.1 ########.10 ########.11 ########.12 ########.13 ########.14 ########.15 ########.16 ########.17 ########.18 ########.2 ########.3 ########.4 ########.5 ########.6 ########.7 ########.8 ########.9 ######.###### ######.##.#### ########.### #####.#### #######.### ####.#######.Y.N ####.##.##### 1 ######.## ######### 1 ###### ####.### 1 #####.###### #####.######## ###.### 1 ###.#### 1 #####.####.#### ###.######### ####.### 1 1 ###.###.### ###.###.###.##### ###.###.###.### ###.###.#### ###.###.####.### ###.###.###.##### #####.######.### #####.###.###### ###########.#### AC ########.#### ###.###.### ')

0 Karma

Builder
0 Karma

Champion

Could you paste the data in pastebin? This will keep its original formatting and we can test it on our systems. I doubt its lost any of the data, its probably just parsing it or rendering it oddly 🙂

0 Karma

Builder

After removing tab spaces I am still loosing my data in splunk 😞

0 Karma

Builder

I think this is bug in Splunk - this same behavior for similar log with multiple tab or spaces.

can you guys please confirm?

0 Karma

Builder

Does anyone get a chance to figure out?

0 Karma

Builder

I am not sure whether this editor will truncate some TAB char or not, if you give me your email address I'll send you text format.

0 Karma

Champion

Could you post some example data with the secret parts hashed out or obscured?

0 Karma

Builder

is this bug in Splunk?

0 Karma

Builder

I can see the data after this line in splunk search

0 Karma

Champion

How did you verify that Splunk has read the file correctly?

0 Karma

Builder

Yes I verified - my log line contains 15 tab char continuously 9-10 times.

due to some privacy reason I can't post entire line here.

0 Karma

Champion

Have you verified that it has read the file correctly or checked for any errors in the internal logs?

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!