Getting Data In

transforms.conf not working as expected

Avantika
Explorer

I have below configurations in transforms and props config files to change the source name of my events from upd:9514 to auditd. But it doesn't seems to be working

Transforms.conf

[change_source_to_auditd]
SOURCE_KEY=MetaData:Source
REGEX= .
DEST_KEY=MetaData:Source
FORMAT=source::auditd

Props.conf

Props.conf

[source::udp:9514]
TRANSFORMS-change_source=change_source_to_auditd

 

 

Below are the sample logs-
Jan 23 19:06:00 172.28.100.238 Jan 23 08:05:18 LIDFP3NTF001.li.local audispd: type=EOE msg=audit(1737619518.941:2165876):
Jan 23 19:06:00 172.28.100.238 Jan 23 08:05:18 LIDFP3NTF001.li.local audispd: type=PROCTITLE msg=audit(1737619518.941:2165876): proctitle=2F7573722F7362696E2F727379736C6F6764002D6E
Jan 23 19:06:00 172.28.100.238 Jan 23 08:05:18 LIDFP3NTF001.li.local audispd: type=SOCKADDR msg=audit(1737619518.941:2165876): saddr=020019727F0000010000000000000000 SADDR={ saddr_fam=inet laddr=127.0.0.1 lport=6514 }
Jan 23 19:06:00 172.28.100.238 Jan 23 08:05:18 LIDFP3NTF001.li.local audispd: type=SYSCALL msg=audit(1737619518.941:2165876): arch=c000003e syscall=42 success=yes exit=0 a0=f a1=7fedf8006c20 a2=10 a3=0 items=0 ppid=1 pid=4564 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm=72733A6D61696E20513A526567 exe="/usr/sbin/rsyslogd" key="network_connect_4" ARCH=x86_64 SYSCALL=connect AUID="unset" UID="root" GID="root" EUID="root" SUID="root" FSUID="root" EGID="root" SGID="root" FSGID="root"
Jan 23 19:06:00 172.28.100.238 Jan 23 08:06:00 LIDFP3NTF002.li.local audispd: type=CRED_DISP msg=audit(1737619560.680:2114873): pid=3709107 uid=985 auid=4294967295 ses=4294967295 msg='op=PAM:setcred grantors=pam_env,pam_localuser,pam_unix acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=? res=success' UID="telegraf" AUID="unset"
Jan 23 19:06:00 172.28.100.238 Jan 23 08:06:00 LIDFP3NTF002.li.local audispd: type=CRED_REFR msg=audit(1737619560.577:2114872): pid=3709107 uid=985 auid=4294967295 ses=4294967295 msg='op=PAM:setcred grantors=pam_env,pam_localuser,pam_unix acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=? res=success' UID="telegraf" AUID="unset"
Jan 23 19:06:00 172.28.100.238 Jan 23 08:06:00 LIDFP3NTF002.li.local audispd: type=USER_ACCT msg=audit(1737619560.577:2114871): pid=3709107 uid=985 auid=4294967295 ses=4294967295 msg='op=PAM:accounting grantors=pam_unix,pam_localuser acct="telegraf" exe="/usr/bin/sudo" hostname=? addr=? terminal=? res=success' UID="telegraf" AUID="unset"
Jan 23 19:06:00 172.28.100.238 Jan 23 08:06:00 LIDFP3NTF002.li.local audispd: type=EOE msg=audit(1737619560.577:2114870):
Jan 23 19:06:00 172.28.100.238 Jan 23 08:06:00 LIDFP3NTF002.li.local audispd: type=PROCTITLE msg=audit(1737619560.577:2114870): proctitle=7375646F002F7573722F7362696E2F706D63002D75002D620031004745542054494D455F5354415455535F4E50
Jan 23 19:06:00 172.28.100.238 Jan 23 08:06:00 LIDFP3NTF002.li.local audispd: type=PATH msg=audit(1737619560.577:2114870): item=0 name="/etc/shadow" inode=132150 dev=fd:00 mode=0100000 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0 OUID="root" OGID="root"
Jan 23 19:06:00 172.28.100.238 Jan 23 08:06:00 LIDFP3NTF002.li.local audispd: type=SYSCALL msg=audit(1737619560.577:2114870): arch=c000003e syscall=257 success=yes exit=9 a0=ffffff9c a1=7fc1d61bbe1a a2=80000 a3=0 items=1 ppid=3709106 pid=3709107 auid=4294967295 uid=985 gid=985 euid=0 suid=0 fsuid=0 egid=985 sgid=985 fsgid=985 tty=(none) ses=4294967295 comm="sudo" exe="/usr/bin/sudo" key="etcpasswd" ARCH=x86_64 SYSCALL=openat AUID="unset" UID="telegraf" GID="telegraf" EUID="root" SUID="root" FSUID="root" EGID="telegraf" SGID="telegraf" FSGID="telegraf"
Jan 23 19:06:00 172.28.100.238 Jan 23 08:06:00 LIDFP3NTF002.li.local audispd: type=EOE msg=audit(1737619560.570:2114869):
Jan 23 19:06:00 172.28.100.238 Jan 23 08:06:00 LIDFP3NTF002.li.local audispd: type=PROCTITLE msg=audit(1737619560.570:2114869): proctitle=7375646F002F7573722F7362696E2F706D63002D75002D620031004745542054494D455F5354415455535F4E50
Jan 23 19:06:00 172.28.100.238 Jan 23 08:06:00 LIDFP3NTF002.li.local audispd: type=PATH msg=audit(1737619560.570:2114869): item=1 name="/lib64/ld-linux-x86-64.so.2" inode=397184 dev=fd:00 mode=0100755 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0 OUID="root" OGID="root"
Jan 23 19:06:00 172.28.100.238 Jan 23 08:06:00 LIDFP3NTF002.li.local audispd: type=PATH msg=audit(1737619560.570:2114869): item=0 name="/usr/bin/sudo" inode=436693 dev=fd:00 mode=0104111 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0 OUID="root" OGID="root"
Jan 23 19:06:00 172.28.100.238 Jan 23 08:06:00 LIDFP3NTF002.li.local audispd: type=EXECVE msg=audit(1737619560.570:2114869): argc=6 a0="sudo" a1="/usr/sbin/pmc" a2="-u" a3="-b" a4="1" a5=4745542054494D455F5354415455535F4E50
Jan 23 19:06:00 172.28.100.238 Jan 23 08:06:00 LIDFP3NTF002.li.local audispd: type=BPRM_FCAPS msg=audit(1737619560.570:2114869): fver=0 fp=0 fi=0 fe=0 old_pp=00000000000000c2 old_pi=00000000000000c2 old_pe=00000000000000c2 old_pa=00000000000000c2 pp=00000000200000c2 pi=00000000000000c2 pe=00000000200000c2 pa=0 frootid=0
Jan 23 19:06:00 172.28.100.238 Jan 23 08:06:00 LIDFP3NTF002.li.local audispd: type=SYSCALL msg=audit(1737619560.570:2114869): arch=c000003e syscall=59 success=yes exit=0 a0=7fe718b344a0 a1=7fe7186addb0 a2=7ffcc797d010 a3=3 items=2 ppid=3709106 pid=3709107 auid=4294967295 uid=985 gid=985 euid=0 suid=0 fsuid=0 egid=985 sgid=985 fsgid=985 tty=(none) ses=4294967295 comm="sudo" exe="/usr/bin/sudo" key="priv_esc" ARCH=x86_64 SYSCALL=execve AUID="unset" UID="telegraf" GID="telegraf" EUID="root" SUID="root" FSUID="root" EGID="telegraf" SGID="telegraf" FSGID="telegraf"

Labels (3)
Tags (1)
0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi @Avantika ,

when I said "located on" I meant in which server, as also @kiran_panchavat pointed on, they must be ocated in the first Heavy Forwarders they are passing through, or (only if there isn't any HF) on the Indexers.

Then, your path, is very strange, it doesn't seem to be a Splunk app, the props.conf and transforms.conf files must be located in this path:

$SPLUNK_HOME/etc/apps/<your_app>/local

Ciao.

Giuseppe

View solution in original post

Avantika
Explorer

It worked for me by adding my file in this path
$SPLUNK_HOME/etc/apps/<your_app>/local

 

Thanks @gcusello @kiran_panchavat 

0 Karma

kiran_panchavat
Motivator

@Avantika 

I implemented the same configuration in my test environment, and it is functioning as expected. Please verify that once the configuration is pushed from the deployment server, it should be available on the heavy forwarder at: `/opt/splunk/etc/apps`.

kiran_panchavat_1-1737702027436.png

 

 

I hope this helps, if any reply helps you, you could add your upvote/karma points to that reply, thanks.

Avantika
Explorer

@kiran_panchavat There is no HF involved

The data is coming via UF with inputs.conf pre configured  with source= udp:9514

could you let me know how are you passing regex in your transforms.conf ? (the goal is to change the name from udp:9514 to auditd)
I tried these regex and didn't got the required results

REGEX = udp\:9514

REGEX = source::udp:9514

 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Avantika ,

where did you locate these conf files?

they must be located in the first full Splunk instance the data pass through, in other words in the first Heavy Forwarder or, if not present, in the Indexers.

Ciao.

Giuseppe

0 Karma

Avantika
Explorer

@gcusello 

Since I'm testing these conf files in my sandpit env first, my conf files are under
etc/apps/git/splunk-deployment-apps/parsing_syslog/local(mylocal branch)

These are the UDP logs and the inputs.conf is configured by user on their end                                                                                                                                                                                                          

0 Karma

isoutamo
SplunkTrust
SplunkTrust

As other already mentioned those configurations must be in first HF/Indexer if there haven't been any HF from path source system (where this input is) to indexers.

If input is in heavy forwarder then those must be there. 

If there are any intermediate HF between UF and Indexers then those must be in this IHF.

If source side is UF and there is nothing else before indexers (or your sandbox) then those must be in your sandbox.

And ensure that those KOs are shared as all apps/system. Otherwise it could be that those are not valid in ingestion phase.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Avantika ,

when I said "located on" I meant in which server, as also @kiran_panchavat pointed on, they must be ocated in the first Heavy Forwarders they are passing through, or (only if there isn't any HF) on the Indexers.

Then, your path, is very strange, it doesn't seem to be a Splunk app, the props.conf and transforms.conf files must be located in this path:

$SPLUNK_HOME/etc/apps/<your_app>/local

Ciao.

Giuseppe

kiran_panchavat
Motivator

@Avantika

Did you deploy the configuration from the deployment server to the heavy forwarder? Is the heavy forwarder connected to the deployment server?

I hope this helps, if any reply helps you, you could add your upvote/karma points to that reply, thanks.
0 Karma
Get Updates on the Splunk Community!

Technical Workshop Series: Splunk Data Management and SPL2 | Register here!

Hey, Splunk Community! Ready to take your data management skills to the next level? Join us for a 3-part ...

Spotting Financial Fraud in the Haystack: A Guide to Behavioral Analytics with Splunk

In today's digital financial ecosystem, security teams face an unprecedented challenge. The sheer volume of ...

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability As businesses scale ...