Trying to chart ONLY the reprocessed cartons.

[jhumkey]

(Data coming from a PLC Conveyor system.)
I'm trying to show how many cartons were RE-processed manually, each day, during the last 7 days.

My first thought, was to subtract count(SCarton)-dc(SCarton) in some sort of eval.
(But days of attempts have shown no luck. Neither by direct subtraction, nor subtraction in an eval, nor subtraction in an eval to variables, before the charting.)

I settled on the following . . .

<dashboard>
  <label>Weekday Carton Count</label>
  <row>
    <panel>
      <title>Cartons processed per day, last Seven Days</title>
      <chart>
        <search>
          <query>source=tcp:5002 Quality!="bad" "Station_Print_Label" |  timechart span=1d count(SCarton),dc(SCarton) </query>
          <earliest>-7d@d</earliest>
          <latest></latest>
        </search>
        <option name="charting.chart">area</option>
        <option name="charting.axisY2.enabled">undefined</option>
      </chart>
    </panel>
  </row>
</dashboard>

At least I can visually see the blue "count" peaks, whenever they exceed the yellow "dc" baseline. Then hover over them for the #'s.

I wish it were ONLY the difference, and, even what I have, I now suspect isn't what I want. For spans of 1d, then if a carton is put back, it's probably put back in the same day. But if this were 5min spans . . . its entirely possible that a carton that first printed in the previous bucket, appears to be unique in the next bucket. So no difference would appear, since the reentry appears in a subsequent bucket.

I've spent days looking. Two major factors hurting me, are A: I'm unclear what's passed via the pipe? Is it a table? Is it matching events left over from any previous comparisons? Is it the sum of all human knowledge? And B: what is the output of an eval? Is it a single scalar value? Is it processed events? Is it a table? Can I even use some sort of eval to subtract dc from count for a given bucket?

If this were C or better Perl . . . I could sweep all the (time sorted) cartons passing the Print Station, and for any serial # I encounter, that has been seen before, I accumulate output for that time bucket, as I now know it's a duplicate. When I reach the end of any time bucket, I'd print the total obtained, and reset the counter to 0 for the next bucket.

But I'm just at a loss how (or if) such a simple comparison (and its chart) is possible in Splunk.

Any pointers would be appreciated.

[jhumkey · Accepted solution]

OP here. I'll post my own answer., in case anyone is looking for the same answer.

source=tcp:5009 Quality!="bad" "PLC_Print_Label_Tag" | timechart span=1h dc(SSCC) as cartons_printed,count(SSCC) as passes | eval manual_reprints=passes-cartons_printed | fields _time manual_reprints cartons_printed
-24h
now

0
350
ellipsisNone
0
visible
visible
visible
linear
linear
undefined
inherit
column
50
10
area
gaps
0.01
default
shiny
all
0
ellipsisMiddle
right
[0x33CCCC,0xCC0033]
stacked

Here's a section of a dashboard. I'm completely unclear on how/why you do the timechart "first" . . . THEN do the subtraction to achieve the result to be posted but . . . the above works. It produced a Stacked Column chart with the cartons per hour on the bottom, and the manually reprinted (human intervention) stacked on top.

One important note. The colors I chose here, show a light blue (cyan?) on the bottom, and Red for the Manual Intervention. This works great but !!!! Changing things above this (other row's) on the Dashboard can INVERT the two colors. As I've edited unrelated row's above this one . . . I've had to flip the color order here. That's irritating, but you can work around it. Good Luck.

[jhumkey]

I picked "code" section . . . but it didn't maintain it. Sorry.

[jhumkey]

I can't show the data (company policy) but no . . . the lines are totally identical except for the date-time stamp. The PLC sends up the exact same carton label, and the Warehouse Mgt Sys responds with the exact same shipping label to print. I'm seeing that (totally same Tag value but with new time) line being responded again, indicating some human placed the carton back in line to be reprinted.

I tried the suggestion above, I get one answer (138) . . . It's just giving me the total # of occurrences of overages across the entire span, not a "per time" division (like "per hour" over a week).

I'll keep looking. Thanks for trying.

[bbartlett_splun]

Can you show us what the data looks like for a regular processed carton, versus a reprocessed carton? Are there any indicators other than a reused serial number?

Without seeing the data, I think it may be as simple as showing only the serial numbers that appear >1 time in whatever time period you're looking for.

for example:
| stats count by serial | where count > 1

From here, you could table or chart whichever fields you're interested in.

To answer your question about what is passed via the pipe: the results of the search before the pipe. If you're familiar with unix/linux command lines, it's the same principle.