Getting Data In

Sourcetype not being applied

gabrsar
Loves-to-Learn Lots

Hi everyone. I'm using Splunk Enterprise (Trial) to understand how things works.

I'm trying to configure some sourcetype for my python/flask application, logs where getting merged incorrectly, with two or more line logs being joined inside a single event and sourcetype is not being applied.

For example, this is a single event in splunk:

[2021-06-23 12:05:09,807] {/program.py:make_request:452} DEBUG - https://localhost:443 "POST /create/user HTTP/1.1" 201 None
[2021-06-23 12:05:09,810] {/program.py:make_request:493} INFO - user created with success id=1234

I also have some logs with this format:

[2021-06-24 17:48:37,490] {/program/main.py:authorize:69} INFO - Host: localhost:5000
  User-Agent: curl/7.64.0
  Accept: */*

I tried creating a new sourcetype on Settings->Data->Source Types. But I noticed two weird things.

1 - If I go on Advanced and configure as I want, It don't save my new regex for LINE_BREAKER. I need it to be: ([\n\r]+)[ but every time I press save, and open again, its the default one ([\n\r]+). If I go on "Events Break" instead and just type my regex it saves. What I'm doing wrong?

2 - It doesn't apply my new sourcetype to my logs. I check on Search->Event lists and my logs are being sourcetyped as output-too_small, now I changed something and it is output-2

Then, I started googling around, and reading some docs, they tell to edit some files on splunk server then I did:

3 - Also tried creating a new sourcetype on $SPUNK_HOME/etc/system/local/props.conf as follow:

[python_flask]
SHOULD_LINEMERGE = False
LINE_BREAKER = ([\n\r]+)\[
NO_BINARY_CHECK = true
disabled = false

4 - Also changed my $SPUNK_HOME/etc/system/local/inputs.conf, and added:

[monitor:///var/log/program/output.log]
sourcetype = python_flask

I restarted with splunk restart both server and universal forwarder, and the only thing that changed is that it started to put sourcetype=output-2 on my events .

I'm quite noob in splunk management, so sorry if any question is dumb, I have already checked the docs, google and so on. Thank you so much in advance.

Labels (2)
0 Karma

codebuilder
Influencer

Check the permissions on the props.conf that you created. I'm going to guess it's owned by root (or another user) and Splunk can't access it. If you are running Splunk as the splunk user you'll need to chmod splunk:splunk $SPUNK_HOME/etc/system/local/props.conf (e.g.) and then cycle Splunk.

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...