Getting Data In

Log file monitoring, wrong parsing

faustf
Communicator

Hy guys,
I've a nodejs application which is logging in a text file in JSON format using the winston library.

As you can see from the image, not all log lines are well parsed:
alt text

There are 2 logs line that are treated as one, and as a result, splunk is not able to detect the fields in the log lines (level, message ....)

This is my configuration:

[monitor:///home/user/myapp/log]
disabled = false
index = myindex
crcSalt = <SOURCE>
sourcetype = json

And this is my log file:

alt text

0 Karma
1 Solution

s2_splunk
Splunk Employee
Splunk Employee

Share your sourcetype definition for "json" in props.conf, please.
If you intended to use the built-in sourcetype for json data, that would be "_json".

View solution in original post

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

Share your sourcetype definition for "json" in props.conf, please.
If you intended to use the built-in sourcetype for json data, that would be "_json".

0 Karma

kmorris_splunk
Splunk Employee
Splunk Employee

I would recommend taking a sample of your data and brining it in through the GUI. When you do this, you can tweak the Breaking, Timestamping, etc... and see how it effects the way the data will be ingested. You can use the BREAK_ONLY_BEFORE parameter to apply a regex that identifies the beginning of each event.

Something like:

BREAK_ONLY_BEFORE       \{\"\w+\":

Once you get the linebreaking correct, save the sourcetype and then start feeding your data in through a forwarder.

0 Karma

faustf
Communicator

Thank you for your answer, but the problem was that the sourcetype was wrong: I wrote json instead o _json, as @ssievert wrote.

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

Glad it works now. Please mark my answer as accepted for posterity. Thanks!

0 Karma
Get Updates on the Splunk Community!

Index This | I’m short for "configuration file.” What am I?

May 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with a Special ...

New Articles from Academic Learning Partners, Help Expand Lantern’s Use Case Library, ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Your Guide to SPL2 at .conf24!

So, you’re headed to .conf24? You’re in for a good time. Las Vegas weather is just *chef’s kiss* beautiful in ...