topic Unique CSV Regex- Process Alarm issue? in Getting Data In

Unique CSV Regex- Process Alarm issue?

teco_akelly — Thu, 13 Jul 2023 20:38:11 GMT

I have raw data file (similar to a CSV) with various records as follows:

Process alarm,5/31/2023,23:19:45,20,11PC_0201,# 2 BOILER STEAM DRUM PRESSURE,Medium,L-Lower Limit,South Main,110.0,110.0,PSIG

Process alarm,5/31/2023,22:45:07,34,25N_0004,CP - 5kV BREAKERALARMS,Medium,TIE-BRKR CLOSED,Common Signal

In my transforms.conf file:

[parseProcessAlarm]
REGEX = Process alarm,([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+)
FORMAT = Date::$1 TimeStamp::$2 TagType::$3 Tagname::$4 Description::$5 AlarmSeverity::$6 AlarmName::$7 SignalCategory::$8 Data::$9 LimitData::$10 EngineeringUnits::$11

My problem is that this expression fails sometimes because a "Process Alarm" record may only have 8 data fields after the initial "Process Alarm" wording as shown above. How can I account for this?

Re: Unique CSV Regex- Process Alarm issue?

gcusello — Fri, 14 Jul 2023 06:26:56 GMT

Hi @teco_akelly,

you should try to configure two regexes for both the cases,

then you should see if at least one regex extracts the correct values, otherwise, you could give a diferent name (e.g. Date1 and Date2) and use eval and coalesce to choose the correct one.

Ciao.

Giuseppe

Re: Unique CSV Regex- Process Alarm issue?

ITWhisperer — Fri, 14 Jul 2023 07:55:01 GMT

Try something like this

[parseProcessAlarm] REGEX = Process alarm,([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+)(,([^,]+),([^,]+),([^,]+)|) FORMAT = Date::$1 TimeStamp::$2 TagType::$3 Tagname::$4 Description::$5 AlarmSeverity::$6 AlarmName::$7 SignalCategory::$8 Data::$10 LimitData::$11 EngineeringUnits::$12

Essentially, make the last three capture groups part of another capture group which has an empty alternative, then bump the last three references by 1 i.e. $10, $11, $12

Re: Unique CSV Regex- Process Alarm issue?

teco_akelly — Wed, 19 Jul 2023 13:52:13 GMT

Is there a way to make the wording "Process alarm" into a field? I have many more other log entry types and would like to make them a searchable field.

Re: Unique CSV Regex- Process Alarm issue?

ITWhisperer — Wed, 19 Jul 2023 15:08:02 GMT

Add a new capture group at the beginning and increment all the references by 1

[parseProcessAlarm] REGEX = (Process alarm),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+)(,([^,]+),([^,]+),([^,]+)|) FORMAT = Event::$1 Date::$2 TimeStamp::$3 TagType::$4 Tagname::$5 Description::$6 AlarmSeverity::$7 AlarmName::$8 SignalCategory::$9 Data::$11 LimitData::$12 EngineeringUnits::$13