There's a few different techniques for combining things like this. The one I think you might find most useful could be... OK, example off some silly data I have. Once I work through that and expla...
See more...
There's a few different techniques for combining things like this. The one I think you might find most useful could be... OK, example off some silly data I have. Once I work through that and explain, I'll make an attempt at doing your searches too. Anyway - Amazon Glacier uploads for my little server, every night it tries to push up new files. I think it's similar enough to your data that the example may work, though forgive me for it being stupidly contrived in so many ways. The idea is there are two messages. One is contains "uploaded part" and the other contains "created an upload_id". I don't have a real good "number" to rex out, but I have a PID I can steal the first two digits of to pretend I have numbers. index="glacier" ( "uploaded part" OR "created an upload_id")
| eval is_actual = if(searchmatch("*created an upload_id*"), 1, 0)
| rex "PID\s+(?<dumb_counter>\d\d)"
| eval is_expected = if(searchmatch("*uploaded part*"), dumb_counter, 0)
| stats sum(dumb_counter) as is_expected, sum(is_actual) as is_actual So looking at that, the first line gets all the data, both types. The second line is using an eval to create "is_actual". And when the event matches 'created an upload_id', that is_actual will be set to 1. Otherwise 0. The third line is a rex, just like yours only more dumb. It creates a field "dumb_counter" which will either be a two digit number, or will be null if it didn't match. (Unfortunately, ALL my lines have a PID, so ... this is broken, but I fix it in the next line using logic just like in line 2. Line 4 then is the fix, where I eval 'is_expected' to either be the dumb_counter I wrote IF the line matches what I need it to match, or 0 if it doesn't. (I don't think you'll need this extra logic, but I do and it was easy enough to explain!) The the last line just adds up the two independently. Afterwords you can easily do a new eval for percent or whatever. We'll do this when we try YOUR search. And it's time for that now. We'll use the same technique, only it'll be messier because you have more conditions to work with. index=index ( ("ProducerClass" AND "*Sending message:*") NOT "*REFRESH*") OR ("OpportunityClass" AND "Processing file: file_name")
| eval is_actual = if(searchmatch("*ProducerClass*") AND searchmatch("*Sending message:*"), 1, 0)
| rex field=_raw "Processing file: file_name with (?<record_count>[^\s]+) records"
| eval is_expected = if(searchmatch("*OpportunityClass*") AND searchmatch("*Processing file: *"), record_count, 0)
| stats sum(is_expected) as is_expected, sum(is_actual) as is_actual
| eval percent = (is_expected / is_actual) *100 Again, line 1 pulls all the data in. (Special note, you use NOT ... which means those records won't be there and we can ignore them in the eval, you'll see! Line 2 creates our is_actual. This line could be left here or moved to after the rex - it won't really matter. Line 3 is our rec to get our record count... Which in line 4 we convert into a new field 'is_expected' ONLY if the event is the right event - this is very, very likely to not be needed, you could extract the field in line 3 with the name 'is_expected', remove this line, and it probably should all work the same. But we're being careful here. The we just sum those in line 5, and do some math in line 6. So of special note! If "file_name" actually stands in for the filename which changes, we'll have to work around that with a wildcard or something. OR if you can drop in a line from each event type (appropriately obfuscated, of course) then we can just work it using one of the other methods. For instance, we may be able to ignore "filename" in the base search, then just edit the rex a wee bit to work around it later, too. Anyhow, give those a shot, and if it works for you (or is easily "fixed" because I'm sure there's some typos in it), then great! Otherwise, let us know what's happening and we can help more. Happy Splunking, Rich