Getting Data In

Sourcetype not being applied

gabrsar
Loves-to-Learn Lots

Hi everyone. I'm using Splunk Enterprise (Trial) to understand how things works.

I'm trying to configure some sourcetype for my python/flask application, logs where getting merged incorrectly, with two or more line logs being joined inside a single event and sourcetype is not being applied.

For example, this is a single event in splunk:

[2021-06-23 12:05:09,807] {/program.py:make_request:452} DEBUG - https://localhost:443 "POST /create/user HTTP/1.1" 201 None
[2021-06-23 12:05:09,810] {/program.py:make_request:493} INFO - user created with success id=1234

I also have some logs with this format:

[2021-06-24 17:48:37,490] {/program/main.py:authorize:69} INFO - Host: localhost:5000
  User-Agent: curl/7.64.0
  Accept: */*

I tried creating a new sourcetype on Settings->Data->Source Types. But I noticed two weird things.

1 - If I go on Advanced and configure as I want, It don't save my new regex for LINE_BREAKER. I need it to be: ([\n\r]+)[ but every time I press save, and open again, its the default one ([\n\r]+). If I go on "Events Break" instead and just type my regex it saves. What I'm doing wrong?

2 - It doesn't apply my new sourcetype to my logs. I check on Search->Event lists and my logs are being sourcetyped as output-too_small, now I changed something and it is output-2

Then, I started googling around, and reading some docs, they tell to edit some files on splunk server then I did:

3 - Also tried creating a new sourcetype on $SPUNK_HOME/etc/system/local/props.conf as follow:

[python_flask]
SHOULD_LINEMERGE = False
LINE_BREAKER = ([\n\r]+)\[
NO_BINARY_CHECK = true
disabled = false

4 - Also changed my $SPUNK_HOME/etc/system/local/inputs.conf, and added:

[monitor:///var/log/program/output.log]
sourcetype = python_flask

I restarted with splunk restart both server and universal forwarder, and the only thing that changed is that it started to put sourcetype=output-2 on my events .

I'm quite noob in splunk management, so sorry if any question is dumb, I have already checked the docs, google and so on. Thank you so much in advance.

Labels (2)
0 Karma

codebuilder
Influencer

Check the permissions on the props.conf that you created. I'm going to guess it's owned by root (or another user) and Splunk can't access it. If you are running Splunk as the splunk user you'll need to chmod splunk:splunk $SPUNK_HOME/etc/system/local/props.conf (e.g.) and then cycle Splunk.

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Community Content Calendar, September edition

Welcome to another insightful post from our Community Content Calendar! We're thrilled to continue bringing ...

Splunkbase Unveils New App Listing Management Public Preview

Splunkbase Unveils New App Listing Management Public PreviewWe're thrilled to announce the public preview of ...

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Are you leveraging automation to its fullest potential in your threat detection strategy?Our upcoming Security ...