Getting Data In

Sourcetype not being applied

gabrsar
Loves-to-Learn Lots

Hi everyone. I'm using Splunk Enterprise (Trial) to understand how things works.

I'm trying to configure some sourcetype for my python/flask application, logs where getting merged incorrectly, with two or more line logs being joined inside a single event and sourcetype is not being applied.

For example, this is a single event in splunk:

[2021-06-23 12:05:09,807] {/program.py:make_request:452} DEBUG - https://localhost:443 "POST /create/user HTTP/1.1" 201 None
[2021-06-23 12:05:09,810] {/program.py:make_request:493} INFO - user created with success id=1234

I also have some logs with this format:

[2021-06-24 17:48:37,490] {/program/main.py:authorize:69} INFO - Host: localhost:5000
  User-Agent: curl/7.64.0
  Accept: */*

I tried creating a new sourcetype on Settings->Data->Source Types. But I noticed two weird things.

1 - If I go on Advanced and configure as I want, It don't save my new regex for LINE_BREAKER. I need it to be: ([\n\r]+)[ but every time I press save, and open again, its the default one ([\n\r]+). If I go on "Events Break" instead and just type my regex it saves. What I'm doing wrong?

2 - It doesn't apply my new sourcetype to my logs. I check on Search->Event lists and my logs are being sourcetyped as output-too_small, now I changed something and it is output-2

Then, I started googling around, and reading some docs, they tell to edit some files on splunk server then I did:

3 - Also tried creating a new sourcetype on $SPUNK_HOME/etc/system/local/props.conf as follow:

[python_flask]
SHOULD_LINEMERGE = False
LINE_BREAKER = ([\n\r]+)\[
NO_BINARY_CHECK = true
disabled = false

4 - Also changed my $SPUNK_HOME/etc/system/local/inputs.conf, and added:

[monitor:///var/log/program/output.log]
sourcetype = python_flask

I restarted with splunk restart both server and universal forwarder, and the only thing that changed is that it started to put sourcetype=output-2 on my events .

I'm quite noob in splunk management, so sorry if any question is dumb, I have already checked the docs, google and so on. Thank you so much in advance.

Labels (2)
0 Karma

codebuilder
SplunkTrust
SplunkTrust

Check the permissions on the props.conf that you created. I'm going to guess it's owned by root (or another user) and Splunk can't access it. If you are running Splunk as the splunk user you'll need to chmod splunk:splunk $SPUNK_HOME/etc/system/local/props.conf (e.g.) and then cycle Splunk.

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...