topic Re: How to change the index at index time in Getting Data In

How to change the index at index time

mataharry — Mon, 21 Jul 2014 22:12:38 GMT

I have data coming from syslog udp:514
but I want to send some events to a different index.

Depending of :

the host they are coming from : ftp or vpn should go to indexB
some keywords in the events. "AUDIT" or "ALERT" should go to indexC

Re: How to change the index at index time

yannK — Mon, 28 Sep 2020 17:08:09 GMT

Method 1 :
The optimal solution is to open several ports, one for each index and have your syslog routed to the proper port.

example :
[udp:514] index=myindexA [udp:515] index=myindexB [udp:516] index=myindexC #etc...
It is also possible to then enforce a sourcetype or a timezone per source .

Method 2: If you cannot separate your inputs, then you have to create props/transforms rules to change the index based on : the content of the events, or the host.

see
in inputs.conf
[udp:514] index=myindexA sourcetype=syslog

in props.conf
To apply only for your input
[source::udp:514] TRANSFORMS-changeindex=routeIndexbasedonhost, routeIndexbasedonevent
OR to apply to all syslog sourcetype

[syslog] TRANSFORMS-changeindex=routeIndexbasedonhost, routeIndexbasedonevent

in transforms.conf

`
[routeIndexbasedonhost]
SOURCE_KEY = MetaData:Host
REGEX=(ftp|vpn)
DEST_KEY=_MetaData:Index
FORMAT=myindexB
WRITE_META=true

[routeIndexbasedonevent]
REGEX=(AUDIT|ALERT)
DEST_KEY=_MetaData:Index
FORMAT=myindexC
WRITE_META=true
`

The order of the transforms is important:
TRANSFORMS-changeindex=routeIndexbasedonhost, routeIndexbasedonevent
will have : myindexA for all, then some will become myindexB, and finally some will become myindexC
so the last one will overwrite the previous ones.

Re: How to change the index at index time

gkanapathy — Mon, 21 Jul 2014 23:13:44 GMT

I would also ask why think you want to send them to different indexes? is it for security reasons? retention reasons?

Re: How to change the index at index time

mataharry — Wed, 23 Jul 2014 16:31:44 GMT

For retention mostly, the vpn logs are voluminous, and I want to keep only 1 month of them.
But sometime for permissions, I want only my security team to be able to see the "ALERT" events.