topic Re: How to get a substr from a fields name in Splunk Search

How to get a substr from a fields name

netanelm7 — Tue, 29 Sep 2020 17:03:31 GMT

Hi everyone,

I want to deliver 2 fields with 1 parameter to a destination panel.
I deliver the string JNL_, the first number contains the first field and the second number contains the second field .
For example "JNL000_01E" (it's in HEXA), the first field name is "JNL000" and the second is "JNL01E".
I want to get the fields "JNL000" and "JNL01E" in the destination panel.
I tried to do that with rex with didn't succeed.
The end goal is to see a timechart with these 2 delivered parameters, my only problem is the rex line.

Thank you!!!

Re: How to get a substr from a fields name

niketn — Mon, 04 Dec 2017 14:46:21 GMT

Is JNL000_01E field name or field value? Same for JNL000 and JNL01E are they going to fields in your data or value?

Can you add some sample data and your current query that you tried?

Re: How to get a substr from a fields name

jplumsdaine22 — Mon, 04 Dec 2017 15:09:05 GMT

I think what you're asking is if you have a result JNL000_01E="foo", you want to create two new fields called parameter1 and parameter2 ? If that's the case this will do the trick.

| foreach JNLA*_* matchseg1="#matchseg1#" matchseg2="#matchseg2#"  [ eval param1="JNL" . "#matchseg1#",param2="JNL" . "#matchseg2#" ]

Here's an example that will run anywhere:

| makeresults 
| eval JNLA000_01E="foo" 
| foreach JNLA*_* matchseg1="#matchseg1#" matchseg2="#matchseg2#"  [ eval param1="JNL" . "#matchseg1#",param2="JNL" . "#matchseg2#" ]

Re: How to get a substr from a fields name

woodcock — Mon, 04 Dec 2017 15:18:20 GMT

Try this:

... | eval output_field = split(input_field, "_")
| eval foo=mvindex(output_field, 0)
| eval bar=mvindex(output_field, 1)

Re: How to get a substr from a fields name

netanelm7 — Tue, 05 Dec 2017 07:17:56 GMT

A field name, thanks for the correction

Re: How to get a substr from a fields name

netanelm7 — Tue, 05 Dec 2017 07:19:58 GMT

Hi jplumsdaine22,

Im sorry for the mistake, i want 2 fields with the name "JNL000" and "JNL01E", not a fields value (Thank you for your answer).

Re: How to get a substr from a fields name

netanelm7 — Tue, 05 Dec 2017 07:20:18 GMT

Im sorry for the mistake, i want 2 fields with the name "JNL000" and "JNL01E", not a fields value (Thank you for your answer).
I've updated the main comment.

Re: How to get a substr from a fields name

woodcock — Tue, 05 Dec 2017 08:25:58 GMT

How do you "deliver the string"? Is this an onclick event, drilldown or what?

Re: How to get a substr from a fields name

netanelm7 — Tue, 05 Dec 2017 09:42:02 GMT

Here's the panels (the main one and the second one):

<panel>
  <table>
    <title>PAIR JNL Preformance Table (Shows the number of times the PAIR JNLs MB/s is greater then 450 MB)</title>
    <search>
      <query>index=storage_18037 sourcetype=csvRotemA_JNL_SUMMARY NOT DATETIME host=RotemA | eval transfer_in_MB=M_JNL_ASYNC_XFER_RATE/1024,IDs="JNL".JOURNAL_ID | where transfer_in_MB&gt;450 | search IDs="JNL000" | dedup _time | timechart span=1h count(transfer_in_MB) by IDs | appendcols [search index=storage_18037 sourcetype=csvRotemA_JNL_SUMMARY NOT DATETIME host=RotemA | eval transfer_in_MB=M_JNL_ASYNC_XFER_RATE/1024,IDs="JNL".JOURNAL_ID | where transfer_in_MB&gt;450 | search IDs="JNL00A" | dedup _time | timechart span=1h count(transfer_in_MB) by IDs] | appendcols [search index=storage_18037 sourcetype=csvRotemA_JNL_SUMMARY NOT DATETIME host=RotemA | eval transfer_in_MB=M_JNL_ASYNC_XFER_RATE/1024,IDs="JNL".JOURNAL_ID | where transfer_in_MB&gt;450 | search IDs="JNL014" | dedup _time | timechart span=1h count(transfer_in_MB) by IDs] | appendcols [search index=storage_18037 sourcetype=csvRotemA_JNL_SUMMARY NOT DATETIME host=RotemA | eval transfer_in_MB=M_JNL_ASYNC_XFER_RATE/1024,IDs="JNL".JOURNAL_ID | where transfer_in_MB&gt;450 | search IDs="JNL01E" | dedup _time | timechart span=1h count(transfer_in_MB) by IDs] | fillnull value="0" JNL000 | fillnull value="0" JNL00A | fillnull value="0" JNL014 | fillnull value="0" JNL01E | eval start_time=_time, end_time=_time+_span | eval JNL000_00A=JNL000+JNL00A, JNL000_014=JNL000+JNL014, JNL000_01E=JNL000+JNL01E, JNL00A_014=JNL00A+JNL014, JNL00A_01E=JNL00A+JNL01E, JNL014_01E=JNL014+JNL01E | fields _time,JNL000_00A,JNL000_014,JNL000_01E,JNL00A_014,JNL00A_01E,JNL014_01E</query>
      <earliest>$timeField1.earliest$</earliest>
      <latest>$timeField1.latest$</latest>
    </search>
    <option name="drilldown">cell</option>
    <drilldown>
      <set token="jnls_mb_counter">$click.name2$</set>
      <eval token="drilldown.earliest">strptime($row._time$,"%Y-%m-%d %H:%M:%S")</eval>
      <eval token="drilldown.latest">strptime($row._time$,"%Y-%m-%d %H:%M:%S") + $row._span$</eval>
    </drilldown>
  </table>
</panel>
<panel depends="$jnls_mb_counter$">
  <table>
    <title>Drilldown Selected JNL MB</title>
    <search>
      <query>index=storage_18037 sourcetype=csvRotemA_JNL_SUMMARY NOT DATETIME host=RotemA | eval transfer_in_MB=M_JNL_ASYNC_XFER_RATE/1024,IDs="JNL".JOURNAL_ID | table _time $jnls_mb_counter$ | eval output_field = split($jnls_mb_counter$, "_") | eval field1=mvindex(output_field, 0) | eval field2="JNL".mvindex(output_field, 1)</query>
      <earliest>$drilldown.earliest$</earliest>
      <latest>$drilldown.latest$</latest>
    </search>
    <option name="drilldown">none</option>
  </table>
</panel>

It's a drilldown

Re: How to get a substr from a fields name

niketn — Tue, 29 Sep 2020 17:04:14 GMT

@netanelm7 can you add couple of data sample from the timechart along with the field names?

Is the splitting of one combined column to two required only for JNL000_01E or other fields like JNL000_00A, JNL014_01E etc as well? If yes, will you be splitting one field at a time based on which column is clicked or all at the same time based on any row clicked?

Re: How to get a substr from a fields name

netanelm7 — Tue, 29 Sep 2020 17:08:29 GMT

There are 6 columns/fields in the main panel.
"JNL000_00A"
"JNL000_014"
"JNL000_01E"
"JNL00A_014"
"JNL00A_01E"
"JNL014_01E"
The person is clicking the value of the column/field in the main panel which triggers a drilldown panel which suppose to display a timechart with 2 lines (the 2 JNLs).
For example, if he clicked a value in "JNL000_00A" column/field, it will display a timechart with 2 lines (one is the values of the field JNL000 and another is the values of the field JNL00A).

I hope i explained it right.

Re: How to get a substr from a fields name

somesoni2 — Tue, 05 Dec 2017 16:04:16 GMT

Change your 2nd panel search with this

index=storage_18037 sourcetype=csvRotemA_JNL_SUMMARY NOT DATETIME host=RotemA | eval transfer_in_MB=M_JNL_ASYNC_XFER_RATE/1024,IDs="JNL".JOURNAL_ID | table _time [| gentimes start=-1 | eval search=replace("$jnls_mb_counter$","(JNL...)_.+","\1") | table search] [| gentimes start=-1 | eval search=replace("$jnls_mb_counter$","(JNL)..._(...)","\1\2") | table search] | timechart max(*) as *

Change the timechart function per your need.

Re: How to get a substr from a fields name

woodcock — Tue, 05 Dec 2017 16:29:51 GMT

In the search for the first panel, add this to the end to create 2 hidden fields to use later:

| foreach JNLA*_*
[ eval _param1="JNL" . "<<MATCHSEG1>>" 
| eval _param2="JNL" . "<<MATCHSEG2>>" ]

Then on click, set 2 tokens, one with $row._param1$ and the other with $row._param2$ .
Then use those tokens in your other search like this

... | fields $token1$ $token2$

Re: How to get a substr from a fields name

niketn — Tue, 05 Dec 2017 18:44:44 GMT

@netanelm7, before I go to the code asked in the question there are several query tuning you should do.

1) Search for search IDs="JNL000" should be in base search i.e. following instead of the | search here

<YourBaseSearch> JOURNAL_ID="000"

2) You are performing two eval in all main search and sub-queries prior to transforming yout data (though timechart command in your case). This is expensive

a) instead of performing eval IDs="JNL".JOURNAL_ID you should run your queries with IDs and after timechart you should use rename. All Journal fields start with 0 so following is what you need.

| rename 0* as JNL*

b) You are performing bytes conversion eval i.e. transfer_in_MB=M_JNL_ASYNC_XFER_RATE/1024. This is not required at all i.e.

| where M_JNL_ASYNC_XFER_RATE>460800
| timechart span=1h count(M_JNL_ASYNC_XFER_RATE) by JOURNAL_ID
| rename  0* as JNL*

Please try out the changes as it should improve the query performance.

Now coming to actual question. Since you have Six fixed columns and based on specific column clicked each column needs to be broken in separate columns. You can use with fields option to match which column is clicked and set the two token for two separate Journals based on Clicked combined Journal.

    <drilldown>
      <condition field="JNL000_00A">
        <eval token="tokFirst">JNL000</eval>
        <eval token="tokSecond">JNL00A</eval>
        <eval token="drilldown.earliest">strptime($row._time$,"%Y-%m-%d %H:%M:%S")</eval>
        <eval token="drilldown.latest">strptime($row._time$,"%Y-%m-%d %H:%M:%S") + $row._span$</eval>
      </condition>
      <condition field="JNL000_014">
        <eval token="tokFirst">JNL000</eval>
        <eval token="tokSecond">JNL014</eval>
        <eval token="drilldown.earliest">strptime($row._time$,"%Y-%m-%d %H:%M:%S")</eval>
        <eval token="drilldown.latest">strptime($row._time$,"%Y-%m-%d %H:%M:%S") + $row._span$</eval>
      </condition>
      <condition field="JNL000_01E">
        <eval token="tokFirst">JNL000</eval>
        <eval token="tokSecond">JNL01E</eval>
        <eval token="drilldown.earliest">strptime($row._time$,"%Y-%m-%d %H:%M:%S")</eval>
        <eval token="drilldown.latest">strptime($row._time$,"%Y-%m-%d %H:%M:%S") + $row._span$</eval>
      </condition>
      <condition field="JNL00A_014">
        <eval token="tokFirst">JNL00A</eval>
        <eval token="tokSecond">JNL014</eval>
        <eval token="drilldown.earliest">strptime($row._time$,"%Y-%m-%d %H:%M:%S")</eval>
        <eval token="drilldown.latest">strptime($row._time$,"%Y-%m-%d %H:%M:%S") + $row._span$</eval>
      </condition>
      <condition field="JNL00A_01E">
        <eval token="tokFirst">mvindex(split("JNL00A_01E","_"),0)</eval>
        <eval token="tokSecond">mvindex(split("JNL00A_01E","_"),1)</eval>
        <eval token="drilldown.earliest">strptime($row._time$,"%Y-%m-%d %H:%M:%S")</eval>
        <eval token="drilldown.latest">strptime($row._time$,"%Y-%m-%d %H:%M:%S") + $row._span$</eval>
      </condition>
      <condition field="JNL014_01E">
        <eval token="tokFirst">JNL014</eval>
        <eval token="tokSecond">JNL01E</eval>
        <eval token="drilldown.earliest">strptime($row._time$,"%Y-%m-%d %H:%M:%S")</eval>
        <eval token="drilldown.latest">strptime($row._time$,"%Y-%m-%d %H:%M:%S") + $row._span$</eval>
      </condition>
    </drilldown>

Although this takes more number of lines of code, if you want to reduce the same, you might have to try Simple XML JavaScript extension or HTML Dashboard.

Re: How to get a substr from a fields name

netanelm7 — Wed, 06 Dec 2017 14:31:34 GMT

This eventually solved the problem, i used this with a combination of "eval token":

  <table>
    <title>PAIR JNL Preformance Table (Shows the number of times the PAIR JNLs MB/s is greater then 450 MB)</title>
    <search>
      <query>index=storage_18037 sourcetype=csvRotemA_JNL_SUMMARY NOT DATETIME host=RotemA | eval transfer_in_MB=M_JNL_ASYNC_XFER_RATE/1024,IDs="JNL".JOURNAL_ID | where transfer_in_MB&gt;450 | search IDs="JNL000" | dedup _time | timechart span=1h count(transfer_in_MB) by IDs | appendcols [search index=storage_18037 sourcetype=csvRotemA_JNL_SUMMARY NOT DATETIME host=RotemA | eval transfer_in_MB=M_JNL_ASYNC_XFER_RATE/1024,IDs="JNL".JOURNAL_ID | where transfer_in_MB&gt;450 | search IDs="JNL00A" | dedup _time | timechart span=1h count(transfer_in_MB) by IDs] | appendcols [search index=storage_18037 sourcetype=csvRotemA_JNL_SUMMARY NOT DATETIME host=RotemA | eval transfer_in_MB=M_JNL_ASYNC_XFER_RATE/1024,IDs="JNL".JOURNAL_ID | where transfer_in_MB&gt;450 | search IDs="JNL014" | dedup _time | timechart span=1h count(transfer_in_MB) by IDs] | appendcols [search index=storage_18037 sourcetype=csvRotemA_JNL_SUMMARY NOT DATETIME host=RotemA | eval transfer_in_MB=M_JNL_ASYNC_XFER_RATE/1024,IDs="JNL".JOURNAL_ID | where transfer_in_MB&gt;450 | search IDs="JNL01E" | dedup _time | timechart span=1h count(transfer_in_MB) by IDs] | fillnull value="0" JNL000 | fillnull value="0" JNL00A | fillnull value="0" JNL014 | fillnull value="0" JNL01E | eval start_time=_time, end_time=_time+_span | eval JNL000_00A=JNL000+JNL00A, JNL000_014=JNL000+JNL014, JNL000_01E=JNL000+JNL01E, JNL00A_014=JNL00A+JNL014, JNL00A_01E=JNL00A+JNL01E, JNL014_01E=JNL014+JNL01E | fields _time,JNL000_00A,JNL000_014,JNL000_01E,JNL00A_014,JNL00A_01E,JNL014_01E</query>
      <earliest>$timeField1.earliest$</earliest>
      <latest>$timeField1.latest$</latest>
    </search>
    <option name="drilldown">cell</option>
    <drilldown>
      <set token="jnls_mb_counter">$click.name2$</set>
      <eval token="drilldown.earliest2">strptime($row._time$,"%Y-%m-%d %H:%M:%S")</eval>
      <eval token="drilldown.latest2">strptime($row._time$,"%Y-%m-%d %H:%M:%S") + $row._span$</eval>
      <eval token="output_field">split($jnls_mb_counter$, "_")</eval>
      <eval token="JNL1_field">mvindex(output_field, 0)</eval>
      <eval token="JNL2_field">mvindex(output_field, 1)</eval>
    </drilldown>
  </table>
</panel>
<panel depends="$jnls_mb_counter$">
  <chart>
    <title>Drilldown Selected PAIR JNL MB</title>
    <search>
      <query>index=storage_18037 sourcetype=csvRotemA_JNL_SUMMARY NOT DATETIME host=RotemA | eval transfer_in_MB=M_JNL_ASYNC_XFER_RATE/1024,IDs="JNL".JOURNAL_ID | where IDs="$JNL1_field$" | timechart avg(transfer_in_MB) span=1m as "$JNL1_field$ Transfer" | appendcols [search index=storage_18037 sourcetype=csvRotemA_JNL_SUMMARY NOT DATETIME host=RotemA | eval transfer_in_MB=M_JNL_ASYNC_XFER_RATE/1024 | where JOURNAL_ID="$JNL2_field$" | timechart avg(transfer_in_MB) span=1m as "JNL$JNL2_field$ Transfer"]</query>
      <earliest>$drilldown.earliest2$</earliest>
      <latest>$drilldown.latest2$</latest>
    </search>
    <option name="charting.chart">line</option>
    <option name="charting.chart.nullValueMode">connect</option>
  </chart>
</panel>

Re: How to get a substr from a fields name

netanelm7 — Thu, 07 Dec 2017 15:42:02 GMT

Thank you very much for your answer!

I've managed to do the first one but the second one (the rename) didn't work.

I solved the problem in a different way, i've posted above how.