All Apps and Add-ons

TA-meraki: How do I remove single quotes from syslog not working with props/transforms?

ColinJacksonPS
Path Finder

I've got data coming in from Meraki APs. It's mostly good, but I can't get rid of the single quotes at index time.

Data:

Nov 15 17:08:27 192.168.255.1 1 1542301707.870571288 main_office events content_filtering_block url='https://baddymcbadface.com/...' category0='Spyware and Adware' server='123.123.123.123:443' client_ip='192.168.204.80'

Here's what I have set up in my props and transforms.conf

Transforms:

[meraki-singlequotes]
REGEX = \s*([^=]+)='([^']*)'
FORMAT = $1::$

Props:

[source:meraki]
REPORT-singlequotes = meraki-singlequotes

I've also tried changing this with [sourcetype:meraki-hq], but that doesn't work as well.

I'm bringing this in via syslog. inputs.conf for this is in the SPLUNK_HOME/etc/apps/launcher/local/ I've added them there, but no dice.
I've also put them in the TA-meraki as well, but it's not taking. Does anyone have any suggestions?

0 Karma
1 Solution

ColinJacksonPS
Path Finder

Here are my updated conf files. Looks like it's still not taking. Thanks @laurie_gellatly and @FrankVl

Props.conf
[source::meraki]
SEDCMD-singlequotes=s/\s*([^=]+)='([^']*)'/g

Transforms.conf (though, I think if I'm doing SEDCMD command, I don't need transforms anymore)
[meraki-singlequotes]
REGEX = \s*([^=]+)='([^']*)'
FORMAT = $1::$2

Anonymized output:

Nov 19 18:42:58 192.168.200.15 1 1542652978.961504488 Board_Room events type=disassociation radio='1' vap='0' client_mac='E4:2B:DD:DD:DD:DD' channel='36' duration='141.595751334' auth_neg_dur='0.839737584' last_auth_ago='140.746016874' is_8021x='1' full_conn='1.309590753' ip_resp='1.309590753' ip_src='192.168.207.2' arp_resp='0.839737584' arp_src='192.168.207.2' dns_server='222.222.222.222'

View solution in original post

0 Karma

ColinJacksonPS
Path Finder

Here are my updated conf files. Looks like it's still not taking. Thanks @laurie_gellatly and @FrankVl

Props.conf
[source::meraki]
SEDCMD-singlequotes=s/\s*([^=]+)='([^']*)'/g

Transforms.conf (though, I think if I'm doing SEDCMD command, I don't need transforms anymore)
[meraki-singlequotes]
REGEX = \s*([^=]+)='([^']*)'
FORMAT = $1::$2

Anonymized output:

Nov 19 18:42:58 192.168.200.15 1 1542652978.961504488 Board_Room events type=disassociation radio='1' vap='0' client_mac='E4:2B:DD:DD:DD:DD' channel='36' duration='141.595751334' auth_neg_dur='0.839737584' last_auth_ago='140.746016874' is_8021x='1' full_conn='1.309590753' ip_resp='1.309590753' ip_src='192.168.207.2' arp_resp='0.839737584' arp_src='192.168.207.2' dns_server='222.222.222.222'
0 Karma

FrankVl
Ultra Champion

That’s not how SEDCMD works. It should be s/<regex matching what needs to be replaced>/<what it needs to be replaced with>/g (the g is to keep applying it, rather than replacing just the first match).

In your case, that could be as simple as s/'//g.

0 Karma

ColinJacksonPS
Path Finder

That did it. Thanks.

0 Karma

laurie_gellatly
Communicator

And remember it won't fix what you've already ingested into the index. Only new ingestions will be altered!

0 Karma

FrankVl
Ultra Champion

I now have a moment to take a closer look and provide a more extensive answer (apart from my earlier comment about REPORT not being performed at indextime), there are several problems with your attempts:

  • A source based stanza in props.conf needs to be defined by [source::<source>], note the double :.
  • If you want to use a sourcetype based stanza instead, it is just [<sourcetype>], no sourcetype: in front of the actual sourcetype.
  • The regex isn't perfect, first fieldname will be the entire start of the event up until the first =; see also: https://regex101.com/r/1LJ1Wl/1
  • as @laurie_gellatly mentions: you're missing a 2 at the end of your FORMAT string.
0 Karma

laurie_gellatly
Communicator

Also your transform needs to $1::$2
And as already stated it happens at search not index time.
If it MUST happen at index then you could look at using sedcmd.

...Laurie:{)

0 Karma

FrankVl
Ultra Champion

A REPORT transform works at searchtime, not at indextime.

Can you perhaps explain a bit further what is and what isn't working, what output do you get now and what is still wrong with that which your are trying to fix?

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...