I am trying to compare threat intelligence data against our web filtering logs. The IOC's come over with http:// or https:// included for each url. Unfortunately my proxy logs do not have this included in the url field and it is not included in the event anywhere at all to concatenate strings. So at search time I am looking at the destination port of each request and determining whether or not it is http or https.
| eval destinationProtocol=case(dest_port ==443, "https", dest_port ==80, "http") | eval url=destinationProtocol. "://" .url
This works as expected when using it in an inline search. However my threat intelligence app is using a data model when running the search to look for matching IOC's and I can't do an EVAL in the data model constraints. So I tried throwing these EVAL's in the props.conf on the search head. Obviously the 2nd EVAL is dependent on the 1st one completing first, so that doesn't work. My understanding is they run in parallel so that will never work.
The other option I tried was to add | eval destinationProtocol=case(dest_port ==443, "https", dest_port ==80, "http") to the TA on the index cluster and add the other eval at search time; but that doesn't seem to work either.
I am at a loss, am I going about this wrong? Any ideas? If anybody has any suggestions I would appreciate it.
End result just needs to be "http://url" in the url field
... View more
