Made partial testing. In the u and addr units, on State field Detected unknown_option validation error. As discussed in Zoom, started writing code for automated unit test to check create/update all units with kOptionsFormatter.

In D529#10843, @erik wrote:

Made partial testing. In the u and addr units, on State field Detected unknown_option validation error. As discussed in Zoom, started writing code for automated unit test to check create/update all units with kOptionsFormatter.

Instead of writing a unit test for each unit by hand you can automate this like so:

1. create a test case file in each module
2. in that test case file create a single test with a data provider
3. data provider would get unit config prefixes of the module (even core), where that unit belongs
4. test itself would accept unit config prefix and:
   • create an empty object (with skip_autoload) of that unit
   • fill required fields with any random stuff (use \kCatDBItemExportHelper::fillRequiredFields method to understand how to do this)
   • attempt to persist that object via kDBItem::Create method
   • assert, that no validation errors array is empty (this way you'll see validation errors in the test suite output)

.

I do not understand relation between test plan and proposed automation logic.

1. As I understand from test plan, not only required fields should be tested, but fields with specific formatter.
2. As I understand from test plan, should test not only create, but also update.
3. Each unit may be specific, may have dependencies to another units, so proposed automation logic seems is not helpful.

In D529#10861, @erik wrote:

I do not understand relation between test plan and proposed automation logic.

1. As I understand from test plan, not only required fields should be tested, but fields with specific formatter.
2. As I understand from test plan, should test not only create, but also update.
3. Each unit may be specific, may have dependencies to another units, so proposed automation logic seems is not helpful.

Try it and see how it goes. If it helps to save time, then it was already worth it. For the complex units, where it won't work, you can create individual tests.

Partially tested (done testing for Core module part).

1. Languages.

1.1. Broken In-Portal clean install due error in lang prefix, preventing language record creation, required in normal installation process.
1.2. Due specific option_sql configuration for PackName and LocalName fields, possible add only record with values, that already exists. So, when no records in Languages table - can't add first language record.

2. Categories - works well in admin interface, but problematic to add through object, because of Priority field options, that are populated only in OnPreSaveCreated event hook.
3. Change logs - sometimes can't create through object same as languages, due specific Prefix and MasterPrefix field options_sql configuration. But in system records are created by direct SQL, so this not a problem. maybe.
4. Users - can't create and update records with US State, because state options are populated too late - after SetFieldsFromHash already set error to the State field.

Found these problems and fixed them (related to tests in the patch):

1. test classes in different modules are named the same, which would result in a fatal error (unable to redeclare a test case class), when executed together
2. test case files aren't placed in the Unit sub folder
3. non-efficient performance-wise:
   • the same user object is created before & deleted after every test
   • the same category objects are loaded before every test
4. identical (at least in values that use the "unique" option in the unit config) base items were created in different tests, and that created issues (failed to create base item), when tests are executed in parallel
5. no protection against failure at base item creation

Found these problems, but wasn't able to fix them:

• missing the test cases for core and modules/custom folders