Unique CSV Regex- Process Alarm issue?

[teco_akelly]

I have raw data file (similar to a CSV) with various records as follows:  

Process alarm,5/31/2023,23:19:45,20,11PC_0201,# 2 BOILER STEAM DRUM PRESSURE,Medium,L-Lower Limit,South Main,110.0,110.0,PSIG

Process alarm,5/31/2023,22:45:07,34,25N_0004,CP - 5kV BREAKERALARMS,Medium,TIE-BRKR CLOSED,Common Signal

In my transforms.conf file:

[parseProcessAlarm]
REGEX = Process alarm,([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+)
FORMAT = Date::$1 TimeStamp::$2 TagType::$3 Tagname::$4 Description::$5 AlarmSeverity::$6 AlarmName::$7 SignalCategory::$8 Data::$9 LimitData::$10 EngineeringUnits::$11

My problem is that this expression fails sometimes because a "Process Alarm" record may only have 8 data fields after the initial "Process Alarm" wording as shown above. How can I account for this?

[teco_akelly]

Is there a way to make the wording "Process alarm" into a field? I have many more other log entry types and would like to make them a searchable field.

[ITWhisperer]

Add a new capture group at the beginning and increment all the references by 1

[parseProcessAlarm]
REGEX = (Process alarm),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+)(,([^,]+),([^,]+),([^,]+)|)
FORMAT = Event::$1 Date::$2 TimeStamp::$3 TagType::$4 Tagname::$5 Description::$6 AlarmSeverity::$7 AlarmName::$8 SignalCategory::$9 Data::$11 LimitData::$12 EngineeringUnits::$13

[ITWhisperer]

Try something like this

[parseProcessAlarm]
REGEX = Process alarm,([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+),([^,]+)(,([^,]+),([^,]+),([^,]+)|)
FORMAT = Date::$1 TimeStamp::$2 TagType::$3 Tagname::$4 Description::$5 AlarmSeverity::$6 AlarmName::$7 SignalCategory::$8 Data::$10 LimitData::$11 EngineeringUnits::$12

Essentially, make the last three capture groups part of another capture group which has an empty alternative, then bump the last three references by 1 i.e. $10, $11, $12

[gcusello]

Hi @teco_akelly,

you should try to configure two regexes for both the cases, 

then you should see if at least one regex extracts the correct values, otherwise, you could give a diferent name (e.g. Date1 and Date2) and use eval and coalesce to choose the correct one.

Ciao.

Giuseppe