Can I use props/transform to make MULTIPLE changes to the same event from a log?

daniel333 — Fri, 09 Feb 2018 20:27:30 GMT

All,

Lets say I have an app log, with a lot going on. I have a certain subset of logs I need to move to a compliance index and change the sourcetype and do a little clean up.

if (event = hello world) then
change sourcetype to "myxactdata"
change index to "compliance"
SED away credit card

I can anyone of these to work, but not all three at once. What's the trick here?

Re: Can I use props/transform to make MULTIPLE changes to the same event from a log?

micahkemp — Fri, 09 Feb 2018 20:37:59 GMT

Once an event is in the parsing queue, changing its sourcetype will not result in the new sourcetype's props/transforms being run on it.

There is a way to do what you're looking for with CLONE_SOURCETYPE. Basically you would clone the hello world event into the sourcetype myxactdata, then drop the event of the original sourcetype. The new sourcetype would have its props/transforms run, so you could change index and use SEDCMD for that sourcetype.

Or, you could use the same REGEX you used to determine you wanted to change the sourcetype to not only change the sourcetype, but also the index, and also run a TRANSFORM on it at index time to accomplish what your SEDCMD did (DEST_KEY = _raw to rewrite _raw).

Edit:

Perhaps your question didn't indicate you were trying to change the sourcetype and use the new sourcetype's props/transforms to perform the extra steps. You can definitely perform multiple index time operations on an event, but make sure the order of operations isn't getting in the way.

For instance, if you use SEDCMD, does your REGEX to set the sourcetype/index no longer match?

topic Re: Can I use props/transform to make MULTIPLE changes to the same event from a log? in Getting Data In

Can I use props/transform to make MULTIPLE changes to the same event from a log?

Re: Can I use props/transform to make MULTIPLE changes to the same event from a log?