Clarify meaning of 'required'

[murphyke]

@aaron0browne I notice that the validator treats 'required' fields as NOT NULL fields. Originally I thought that a 'required' column should be transmitted and populated as possible but not necessarily be NOT NULL - hence the separate existence of NOT NULL constraints in the data model. It doesn't look like the validator pays attention to the NOT NULL constraints. Is this the wave of the future, and if so, should we get rid of the separate NOT NULL constraints in the data-models repo?

[gracebrownecodes]

I think the difference between NOT NULL and "required" is (and always has been) unclear.

I argued from the beginning that all fields should be "required" (in that they must be sent in a data submission, even if they are null), and I believe we are moving in that direction.

However, the validator should definitely be using the NOT NULL constraints instead of the required attribute for this check. Unfortunately, this means passing an entirely different piece of the data models client information into the TableValidator and the FieldValidators below it. I believe the relevant sections of code (working up from the actual validation code) are here, here, and here.

[bruth]

Is this issue still relevant? If so, can it be clarified?

[murphyke]

The requirement is that the validator needs to check the NOT NULL constraints instead of the murky "required" attribute, and AFAIK the issue is still relevant.