Unexpected output: Affected paths: relative - resubmit patch.
⚙ D494 INP-1863 - Correct PHPUnit integration
Page MenuHomeIn-Portal Phabricator

INP-1863 - Correct PHPUnit integration
ClosedPublic

Authored by alex on Nov 8 2024, 1:11 PM.

Details

Test Plan
IMPORTANT: Apply patch using arc patch D494 command, because the Phabricator-generated patch doesn't correctly display file moves, that have no content changes.

Preparations

  1. apply patch from the D496 and D495 (update from 27-Nov-2024: they're committed already; just do svn update)
  2. if you don't have Selenium Server installed/running, then install/run it using instructions from the Preparations section of the D451
  3. in IDE:
    • in the core/tests/Functional folder create CoreWorkflowTest.php file with this content:
<?php
class CoreWorkflowTest extends AbstractBrowserTestCase
{

    public function testSample()
    {
        $session = $this->getSession();

        $url = $this->Application->HREF('platform/login/register', '', array('no_amp' => 1));
        $session->visit($url);

        $this->assertTrue(true);
    }

}
    • apply changes made (after the patch was applied) to the tools/build/build.properties.dist file onto the tools/build/build.properties file
  1. remove contents of the core/units/helpers/xml_helper.php and core/units/helpers/xml_helper5.php files
  2. in the "tools/build/targets/common.xml" file add these lines after <arg path="${phpunit.config.file}"/> line (4 places) <arg value="--testdox"/> line
  3. in the tools/run_event.php file comment out the exit_code(1, 'This script needs to be enabled manually !'); line
  4. in CLI:
    • run the composer install command, to actualize vendor folder content
    • run the phing -f tools/build/build_custom.xml db-dump command to create a dump
    • for each of the build-*.sql files created in the system directory:
      • remove the mysqldump: [Warning] Using a password on the command line interface can be insecure. line (if found)
      • save changes

Part 1 - the PHPUnit isn't used in the default build workflow

  1. delete the "build" folder
  2. run the phing -f tools/build/build_all.xml build command
  3. confirm, by looking at the colorized output of the above command, that phpunit-ci or phpunit targets weren't called
  4. delete the "build" folder
  5. run the phing -f tools/build/build_custom.xml build command
  6. confirm, by looking at the colorized output of the above command, that phpunit-ci or phpunit targets weren't called

Part 2 - core test suite structure testing (via phpunit itself)

  1. run the vendor/bin/phpunit -c tools/build/inc/phpunit.xml --testdox command
  2. confirm, that the output looks similar to this:
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.

Runtime:       PHP 5.6.40 with Xdebug 2.5.5
Configuration: .../tools/build/inc/phpunit.xml

kFormatter
...

SampleUnit
...

CoreWorkflow
...

SampleWorkflow
...
  1. run the vendor/bin/phpunit -c tools/build/inc/phpunit.xml --testdox --testsuite Unit command
  2. confirm, that the output looks similar to this:
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.

Runtime:       PHP 5.6.40 with Xdebug 2.5.5
Configuration: .../tools/build/inc/phpunit.xml

kFormatter
...

SampleUnit
...
  1. run the vendor/bin/phpunit -c tools/build/inc/phpunit.xml --testdox --testsuite Functional command
  2. confirm, that the output looks similar to this:
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.

Runtime:       PHP 5.6.40 with Xdebug 2.5.5
Configuration: .../tools/build/inc/phpunit.xml

CoreWorkflow
...
SampleWorkflow
...

Part 3 - custom test suite structure testing (via phpunit itself)

  1. run the vendor/bin/phpunit -c tools/build/inc/custom_phpunit.xml --testdox command
  2. confirm, that the output looks similar to this:
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.

Runtime:       PHP 5.6.40 with Xdebug 2.5.5
Configuration: .../tools/build/inc/phpunit.xml

SampleUnit
...

SampleWorkflow
...
  1. run the vendor/bin/phpunit -c tools/build/inc/custom_phpunit.xml --testdox --testsuite Unit command
  2. confirm, that the output looks similar to this:
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.

Runtime:       PHP 5.6.40 with Xdebug 2.5.5
Configuration: .../tools/build/inc/phpunit.xml

SampleUnit
...
  1. run the vendor/bin/phpunit -c tools/build/inc/custom_phpunit.xml --testdox --testsuite Functional command
  2. confirm, that the output looks similar to this:
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.

Runtime:       PHP 5.6.40 with Xdebug 2.5.5
Configuration: .../tools/build/inc/phpunit.xml

SampleWorkflow
...

Part 4 - core test suite structure testing (via phing's "phpunit" task)

  1. delete the "build" folder
  2. run the phing -f tools/build/build_all.xml phpunit command
  3. confirm, that "build/logs/clover.xml" and "build/logs/junit.xml" files doesn't exist
  4. confirm, by looking at the colorized output of the above command, that phpunit target was called
  5. confirm, that the output looks similar to this:
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.

Runtime:       PHP 5.6.40 with Xdebug 2.5.5
Configuration: /mnt/UserData/home/alex/web/d/in-portal.52x/tools/build/inc/phpunit.xml

kFormatter
...

SampleUnit
...

CoreWorkflow
...

SampleWorkflow
...
  1. delete the "build" folder
  2. run the phing -f tools/build/build_all.xml phpunit -Dphpunit.testsuite=Unit command
  3. confirm, that "build/logs/clover.xml" and "build/logs/junit.xml" files doesn't exist
  4. confirm, by looking at the colorized output of the above command, that phpunit target was called
  5. confirm, that the output looks similar to this:
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.

Runtime:       PHP 5.6.40 with Xdebug 2.5.5
Configuration: /mnt/UserData/home/alex/web/d/in-portal.52x/tools/build/inc/phpunit.xml

kFormatter
...

SampleUnit
...
  1. delete the "build" folder
  2. run the phing -f tools/build/build_all.xml phpunit -Dphpunit.testsuite=Functional command
  3. confirm, that "build/logs/clover.xml" and "build/logs/junit.xml" files doesn't exist
  4. confirm, by looking at the colorized output of the above command, that phpunit target was called
  5. confirm, that the output looks similar to this:
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.

Runtime:       PHP 5.6.40 with Xdebug 2.5.5
Configuration: /mnt/UserData/home/alex/web/d/in-portal.52x/tools/build/inc/phpunit.xml

CoreWorkflow
...

SampleWorkflow
...

Part 5 - core test suite structure testing (via phing's "phpunit-ci" task)

  1. delete the "build" folder
  2. run the phing -f tools/build/build_all.xml phpunit-ci command
  3. confirm, that "build/logs/clover.xml" and "build/logs/junit.xml" files were created and they're not empty
  4. confirm, by looking at the colorized output of the above command, that phpunit-ci target was called
  5. confirm, that the output looks similar to this:
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.

Runtime:       PHP 5.6.40 with Xdebug 2.5.5
Configuration: /mnt/UserData/home/alex/web/d/in-portal.52x/tools/build/inc/phpunit.xml

kFormatter
...

SampleUnit
...

CoreWorkflow
...

SampleWorkflow
...

Generating code coverage report in Clover XML format ... done
  1. delete the "build" folder
  2. run the phing -f tools/build/build_all.xml phpunit-ci -Dphpunit.testsuite=Unit command
  3. confirm, that "build/logs/clover.xml" and "build/logs/junit.xml" files were created and they're not empty
  4. confirm, by looking at the colorized output of the above command, that phpunit-ci target was called
  5. confirm, that the output looks similar to this:
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.

Runtime:       PHP 5.6.40 with Xdebug 2.5.5
Configuration: /mnt/UserData/home/alex/web/d/in-portal.52x/tools/build/inc/phpunit.xml

kFormatter
...

SampleUnit
...


Generating code coverage report in Clover XML format ... done
  1. delete the "build" folder
  2. run the phing -f tools/build/build_all.xml phpunit-ci -Dphpunit.testsuite=Functional command
  3. confirm, that "build/logs/clover.xml" and "build/logs/junit.xml" files were created and they're not empty
  4. confirm, by looking at the colorized output of the above command, that phpunit-ci target was called
  5. confirm, that the output looks similar to this:
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.

Runtime:       PHP 5.6.40 with Xdebug 2.5.5
Configuration: /mnt/UserData/home/alex/web/d/in-portal.52x/tools/build/inc/phpunit.xml

CoreWorkflow
...

SampleWorkflow
...


Generating code coverage report in Clover XML format ... done

Part 6 - custom test suite structure testing (via phing's "phpunit" task)

  1. delete the "build" folder
  2. run the phing -f tools/build/build_custom.xml phpunit command
  3. confirm, that "build/logs/clover.xml" and "build/logs/junit.xml" files doesn't exist
  4. confirm, by looking at the colorized output of the above command, that phpunit target was called
  5. confirm, that the output looks similar to this:
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.

Runtime:       PHP 5.6.40 with Xdebug 2.5.5
Configuration: /mnt/UserData/home/alex/web/d/in-portal.52x/tools/build/inc/custom_phpunit.xml

SampleUnit
...

SampleWorkflow
...
  1. delete the "build" folder
  2. run the phing -f tools/build/build_custom.xml phpunit -Dphpunit.testsuite=Unit command
  3. confirm, that "build/logs/clover.xml" and "build/logs/junit.xml" files doesn't exist
  4. confirm, by looking at the colorized output of the above command, that phpunit target was called
  5. confirm, that the output looks similar to this:
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.

Runtime:       PHP 5.6.40 with Xdebug 2.5.5
Configuration: /mnt/UserData/home/alex/web/d/in-portal.52x/tools/build/inc/custom_phpunit.xml

SampleUnit
...
  1. delete the "build" folder
  2. run the phing -f tools/build/build_custom.xml phpunit -Dphpunit.testsuite=Functional command
  3. confirm, that "build/logs/clover.xml" and "build/logs/junit.xml" files doesn't exist
  4. confirm, by looking at the colorized output of the above command, that phpunit target was called
  5. confirm, that the output looks similar to this:
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.

Runtime:       PHP 5.6.40 with Xdebug 2.5.5
Configuration: /mnt/UserData/home/alex/web/d/in-portal.52x/tools/build/inc/custom_phpunit.xml

SampleWorkflow
...

Part 7 - custom test suite structure testing (via phing's "phpunit-ci" task)

  1. delete the "build" folder
  2. run the phing -f tools/build/build_custom.xml phpunit-ci command
  3. confirm, that "build/logs/clover.xml" and "build/logs/junit.xml" files were created and they're not empty
  4. confirm, by looking at the colorized output of the above command, that phpunit-ci target was called
  5. confirm, that the output looks similar to this:
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.

Runtime:       PHP 5.6.40 with Xdebug 2.5.5
Configuration: /mnt/UserData/home/alex/web/d/in-portal.52x/tools/build/inc/custom_phpunit.xml

SampleUnit
...

SampleWorkflow
...

Generating code coverage report in Clover XML format ... done
  1. delete the "build" folder
  2. run the phing -f tools/build/build_custom.xml phpunit-ci -Dphpunit.testsuite=Unit command
  3. confirm, that "build/logs/clover.xml" and "build/logs/junit.xml" files were created and they're not empty
  4. confirm, by looking at the colorized output of the above command, that phpunit-ci target was called
  5. confirm, that the output looks similar to this:
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.

Runtime:       PHP 5.6.40 with Xdebug 2.5.5
Configuration: /mnt/UserData/home/alex/web/d/in-portal.52x/tools/build/inc/custom_phpunit.xml

SampleUnit
...


Generating code coverage report in Clover XML format ... done
  1. delete the "build" folder
  2. run the phing -f tools/build/build_custom.xml phpunit-ci -Dphpunit.testsuite=Functional command
  3. confirm, that "build/logs/clover.xml" and "build/logs/junit.xml" files were created and they're not empty
  4. confirm, by looking at the colorized output of the above command, that phpunit-ci target was called
  5. confirm, that the output looks similar to this:
PHPUnit 5.7.27 by Sebastian Bergmann and contributors.

Runtime:       PHP 5.6.40 with Xdebug 2.5.5
Configuration: /mnt/UserData/home/alex/web/d/in-portal.52x/tools/build/inc/custom_phpunit.xml

SampleWorkflow
...


Generating code coverage report in Clover XML format ... done

Testing (part 8 - the "/system/mink_config.php" requirement)

  1. move the "mink_config.php" file away from the "/system" folder
  2. run the vendor/bin/phpunit -c tools/build/inc/phpunit.xml command
  3. confirm, that command now fails with "The Mink configuration file does not exist." message (via thrown exception)
  4. move the "mink_config.php" file back to "/system" folder

Diff Detail

Repository
rINP In-Portal
Branch
/in-portal/branches/5.2.x
Lint
Lint OK
Unit
No Unit Test Coverage
Build Status
Buildable 1326
Build 1326: arc lint + arc unit

Event Timeline

alex created this revision.Nov 8 2024, 1:11 PM
alex requested review of this revision.Nov 8 2024, 1:11 PM
Harbormaster completed remote builds in B1325: Diff 1275.
alex updated this revision to Diff 1276.Nov 8 2024, 1:20 PM
  1. Updating D494: INP-1863 - Correct PHPUnit integration #
  2. Enter a brief description of the changes included in this update.
  3. The first line is used as subject, next lines as comment. #
  4. Included changes:
  5. M composer.json
  6. M composer.lock
  7. M core/kernel/application.php
  8. V core/tests/AbstractTestCase.php to: core/tests/Unit/AbstractTestCase.php
  9. V core/tests/Unit/AbstractTestCase.php from: core/tests/AbstractTestCase.php
  10. V core/tests/Unit/kernel/utility/formatters/kFormatterTest.php from: core/tests/kernel/utility/formatters/kFormatterTest.php
  11. V core/tests/kernel/utility/formatters/kFormatterTest.php to: core/tests/Unit/kernel/utility/formatters/kFormatterTest.php
Harbormaster completed remote builds in B1326: Diff 1276.
alex updated this revision to Diff 1277.Nov 8 2024, 1:33 PM

Included moved files, that were missing in original diff.

alex edited the test plan for this revision. (Show Details)Nov 8 2024, 1:48 PM
alex updated this revision to Diff 1278.Nov 9 2024, 8:51 AM

Same diff, but file moves in "modules/custom" folder are displayed correctly.

alex edited the test plan for this revision. (Show Details)Nov 9 2024, 9:11 AM
alex edited the summary of this revision. (Show Details)Nov 9 2024, 9:16 AM
alex edited the summary of this revision. (Show Details)

Non-obvious Phabricator Differential logic:

  • the arc patch command svn mv SVN command (file move) as svn cp (file copy) + svn rm (file removal) commands;
  • directory removal, where file move occurred, isn't marked as removed.
erik added a comment.Nov 11 2024, 8:42 AM

Partially tested, tests from part 4 like "confirm, by looking at the colorized output of the above command, that phpunit target was called" - failed. No PHPunit output generated.

alex added a comment.Nov 11 2024, 8:47 AM
In D494#10041, @erik wrote:

Partially tested, tests from part 4 like "confirm, by looking at the colorized output of the above command, that phpunit target was called" - failed. No PHPunit output generated.

@erik , was the build itself successful?

erik added a comment.Nov 11 2024, 8:50 AM

No, build was not successful.

[phingcall] /home/sites/dev/web/SVN/5.2.x/tools/build/targets/deploy.xml:37:41: Task exited with code 1

BUILD FAILED
/home/sites/dev/web/SVN/5.2.x/tools/build/targets/deploy.xml:25:6: /home/sites/dev/web/SVN/5.2.x/tools/build/targets/deploy.xml:29:22: Execution of the target buildfile failed. Aborting.

Total time: 29.8418 seconds

alex edited the test plan for this revision. (Show Details)Nov 11 2024, 11:20 AM
alex added a comment.Nov 11 2024, 11:22 AM
In D494#10043, @erik wrote:

No, build was not successful.

[phingcall] /home/sites/dev/web/SVN/5.2.x/tools/build/targets/deploy.xml:37:41: Task exited with code 1

BUILD FAILED
/home/sites/dev/web/SVN/5.2.x/tools/build/targets/deploy.xml:25:6: /home/sites/dev/web/SVN/5.2.x/tools/build/targets/deploy.xml:29:22: Execution of the target buildfile failed. Aborting.

Total time: 29.8418 seconds

I've added this item to the Preparations part of the test plan:

5. in the `tools/run_event.php` file comment out the `exit_code(1, 'This script needs to be enabled manually !');` line

Now the build should be successful.

erik added a comment.Nov 12 2024, 4:36 AM

Can't test because of different php version on VM.

erik added a comment.Nov 12 2024, 10:21 AM

tested, each test command, containing build_all.xml or build_custom.xml leads to config.php change where HMAC key containing variable ($_CONFIG['Misc']['SecurityHmacKey']) disappears, and then build failed due [InvalidArgumentException] "The HMAC key is empty."

Image with typical console output is attached.

alex edited the test plan for this revision. (Show Details)Nov 13 2024, 10:15 AM
alex edited the test plan for this revision. (Show Details)
alex edited the test plan for this revision. (Show Details)
alex added a comment.Nov 17 2024, 10:56 AM
In D494#10047, @erik wrote:

tested, each test command, containing build_all.xml or build_custom.xml leads to config.php change where HMAC key containing variable ($_CONFIG['Misc']['SecurityHmacKey']) disappears, and then build failed due [InvalidArgumentException] "The HMAC key is empty."

Image with typical console output is attached.

@erik , the fix for this issue was done in the D495. Please test that differential revision alone & in combination with this differential revision. Then decide if changes are needed to either of the differential revisions mentioned.

alex edited the test plan for this revision. (Show Details)Nov 20 2024, 5:50 AM
alex updated this revision to Diff 1293.Nov 22 2024, 5:03 AM

Rebasing.

erik added a comment.Mon, Nov 25, 5:34 AM

Partialy tested, build failed after "run the phing -f tools/build/build_all.xml phpunit command" in the part 4 of test plan

alex edited the test plan for this revision. (Show Details)Mon, Nov 25, 6:00 AM
alex updated this revision to Diff 1294.EditedMon, Nov 25, 6:36 AM
  1. fixed sh: ${phpunit.testsuite}: bad substitution error
  2. use proper php binary for executing PHPUnit
alex updated this revision to Diff 1296.Tue, Nov 26, 2:54 AM

Speed up code coverage collection 4.25x times by not building coverage report for files, that weren't executed in tests.

alex updated this revision to Diff 1297.Tue, Nov 26, 12:01 PM

Removed debug code, that added from test plan.

erik added a comment.Wed, Nov 27, 5:11 AM

Failed build at part 4 after command

phing -f tools/build/build_all.xml phpunit

BUILD FAILED
/home/sites/dev/web/SVN/5.2.x/tools/build/targets/common.xml:192:6: /home/sites/ dev/web/SVN/5.2.x/tools/build/targets/common.xml:205:46: Task exited with code 2

Total time: 6 minutes 42.55 seconds

erik requested changes to this revision.Wed, Nov 27, 5:45 AM

Same way failed all builds until part 5.7 tests continued, because all formal requirements passed.

command from 5.7

phing -f tools/build/build_all.xml phpunit-ci -Dphpunit.testsuite=Unit command

had another specific error

build_all.xml phpunit-ci -Dphpunit.testsuite=Unit
Buildfile: tools/build/build_all.xml is not readable!

so, failed all formal requirements after this part (5.8, 5.9, 5.10)

This revision now requires changes to proceed.Wed, Nov 27, 5:45 AM
alex added a comment.Wed, Nov 27, 7:40 AM
In D494#10195, @erik wrote:

Failed build at part 4 after command

phing -f tools/build/build_all.xml phpunit

BUILD FAILED
/home/sites/dev/web/SVN/5.2.x/tools/build/targets/common.xml:192:6: /home/sites/ dev/web/SVN/5.2.x/tools/build/targets/common.xml:205:46: Task exited with code 2

Total time: 6 minutes 42.55 seconds

I have no idea what causes this (the build log on the screenshot is incomplete), but probably the Selenium Server wasn't running, because couple of PHPUnit tests can' run 6 minutes and that suggests some kind of a timeout error. Please carefully follow the test plan (including preparations phase).

In D494#10196, @erik wrote:

Same way failed all builds until part 5.7 tests continued, because all formal requirements passed.

command from 5.7

phing -f tools/build/build_all.xml phpunit-ci -Dphpunit.testsuite=Unit command

had another specific error

build_all.xml phpunit-ci -Dphpunit.testsuite=Unit
Buildfile: tools/build/build_all.xml is not readable!

so, failed all formal requirements after this part (5.8, 5.9, 5.10)

This is happening, because you're running the command in the build/logs folder instead of the project's root folder.

alex requested review of this revision.Wed, Nov 27, 7:40 AM
alex edited the test plan for this revision. (Show Details)Wed, Nov 27, 7:52 AM
erik accepted this revision.Fri, Nov 29, 4:36 AM

All tested, All formal checks passed. But all builds, that has call to selenium - failed with message like.

BUILD FAILED
/home/sites/dev/web/SVN/5.2.x/tools/build/targets/common.xml:216:6: /home/sites/dev/web/SVN/5.2.x/tools/build/targets/common.xml:235:46: Task exited with code 2

Total time: 31.6997 seconds

At selenium window also some errors displayed -

This revision is now accepted and ready to land.Fri, Nov 29, 4:36 AM
alex added a comment.Fri, Nov 29, 5:42 AM
In D494#10203, @erik wrote:

All tested, All formal checks passed. But all builds, that has call to selenium - failed with message like.

BUILD FAILED
/home/sites/dev/web/SVN/5.2.x/tools/build/targets/common.xml:216:6: /home/sites/dev/web/SVN/5.2.x/tools/build/targets/common.xml:235:46: Task exited with code 2

Total time: 31.6997 seconds

At selenium window also some errors displayed -

Discussed directly with Erik. The Selenium tests failed because of 2 problems:

  1. The Firefox version was outdated and therefore GeckoDriver was only opening new Browser Window, but wasn't able to perform any actions in it
  2. The /system/config.php file had localhost in it's $_CONFIG['Misc']['Domain'] key and therefore http://localhost/ URL was visited instead of a proper URL

Now everything is fixed.

This revision was automatically updated to reflect the committed changes.