Ask Reuben

Optimistic vs Pessimistic Field Validation

How do I make sure that ALL the values entered in an INPUT are correct?

Most of you should be aware of the concepts of optimistic and pessimistic locking as it relates to the concept of how to handle concurrent access to the same database records (if not then that will be a future Ask-Reuben).  When it comes to validating the data entered by an INPUT dialog, there are two coding patterns that can be used to validate the data that I like also to describe as being pessimistic and optimistic.


Optimistic field validation is typically found in older code and has the following properties …

  • IF I know that the fields in the dialog were valid at the beginning of the dialog
  • AND I use logic inside ON CHANGE (or AFTER FIELD for code written before ON CHANGE was implemented) to validate that the new field value is correct.

… then I “know” that the field values after the INPUT has completed are valid and these can be safely written to the database.

This approach has flaws…

  1. It is not guaranteed that the field values are valid at the beginning of the dialog.  If a field has an invalid value at the beginning and the user does not touch the field then no logic will occur to validate and thus identify an invalid field value.  It is possible to have invalid values by invalid setup e.g. invalid default values.
  2. Logic inside an executed block could change the value of a field.  These blocks could be any block that can potentially be triggered in the middle of a dialog such as ON CHANGE, AFTER FIELD, BEFORE FIELD, ON ACTION.  If the user does not subsequently visit and touch that field to trigger that changed fields ON CHANGE or AFTER FIELD then no logic will occur to check that the new field value is valid.

Pessimistic field validation as the name suggests takes an approach that considers that your logic may not be perfect and closes that gap by explicitly validating every field in the dialog as part of the AFTER INPUT.

Code that is pessimistic will have the following pattern where the validation that occurs after every field in the AFTER FIELD block …

AFTER FIELD field_name 
    CALL field_name_valid() RETURNING ok, error_text
    IF ok THEN
        # carry on 
    ELSE
        ERROR error_text
        NEXT FIELD field_name
    END IF

… will be repeated in the AFTER INPUT block for every single field.

AFTER INPUT
    CALL field_name_1_valid() RETURNING ok, error_text
    IF ok THEN
        # carry on 
    ELSE
        ERROR error_text
        NEXT FIELD field_name_1
    END IF
    CALL field_name_2_valid() RETURNING ok, error_text
    IF ok THEN
        # carry on 
    ELSE
        ERROR error_text
        NEXT FIELD field_name_2
    END IF
    ...
    CALL field_name_N_valid() RETURNING ok, error_text
    IF ok THEN
        # carry on 
    ELSE
        ERROR error_text
        NEXT FIELD field_name_N
     END IF

The detractors for this approach will make comments such as …

  • that is a lot of unnecessary typing
  • field validation is being repeated unnecessarily

To cut down on typing then the pre-processor is a valuable tool.  You will notice that in my code above there is lot of little repeated code pattern where the only thing that differs is the field name.

If you define a macro such as

&define field_valid(p1) \
    CALL p1 ## _valid() \
    IF ok THEN ELSE ERROR error_text NEXT FIELD p1 END IF

… then your code becomes …

AFTER FIELD field1 
    field_valid(field1)

AFTER FIELD field2
    field_valid(field2)

...
AFTER FIELD fieldN
    field_valid(fieldN)

AFTER INPUT
    field_valid(field1)
    field_valid(field2)
    ...
    field_valid(fieldN)

… and a junior developer is more efficient as adding a field is copying and pasting 1 line and changing the macro parameter.

The key things to note in the pre-processor macro are the use of ## for concatenation, and the use of a macro parameter.

It is true that the field validation is repeated twice.  For instance the field that the user is in will be validated twice if the user accepts the dialog as the AFTER FIELD for that field will validate the field and again in the AFTER INPUT the field will be validated.  Field validation is generally cheap but if the field validation logic is complex and noticeable gains could be made by only executing that logic once then logic can be inserted to keep track of a fields value when it was last validated and only doing the field validation again when the value changes.  Something like …

DEFINE last DICTIONARY OF STRING
...
IF rec.field_name = last["field_name"] THEN -- value hasn't changed
   LET ok = TRUE
ELSE -- value has changed
   CALL field_name_valid() RETURNING ok, error_text
END IF
IF ok THEN
   # carry on
   LET last["field_name"] = rec.field_name -- record value that validation occurred for
ELSE
   ERROR error_text
   NEXT FIELD field_name
END IF

… using a DICTIONARY is a handy way of keeping track of things at a field level by using the field name as the key to the DICTIONARY variable.

By using the pre-processor this logic can be refined and hidden from the developer inside the pre-processor macro.  Similarly if you decide to implement warnings that can be overridden, these too can also be implemented inside a macro.  A warning would require that you keep track of the value that triggered a warning and the fact that the warning was acknowledged and overridden.


I tend to see the optimistic pattern inside older code.  Developers assume that the defaults are good or that their code logic changes a variable to a valid value, when the passage of time shows that a default value is not always valid or that code changes a variable value in the middle of a dialog.

Wether you choose a pessimistic or an optimistic approach is upto you.  An optimistic approach requires you to be disciplined but it faces the danger that once an invalid value gets through then nothing will capture it.  A pessimistic approach will capture scenarios where your developers or your users have left a hole that allows an invalid value can get through, at the cost of some extra CPU cycles as some validation occurs twice.