The Codecov service is a way to monitor test vs code coverage of a project over time and allows for the code coverage % + delta to be reported in each PR.
This commits:
* Adds a Codecov configuration.
* Adds a convenience script to the `composer.json` file to run the tests with or without code coverage.
* Adds a new matrix variable to the GH Actions `test` workflow to run the tests with code coverage and send the results to the Codecov service.
Notes:
- This disables the code coverage reporting in the "normal" test runs, including disabling `xdebug` for those runs which should make them slightly faster.
- This splits the test runs into two sets:
* High/low PHP are being run with code coverage (and have been removed from the "normal" test run matrix).
* For all other PHP versions, the tests are being run without code coverage.
* Adds a badge to the README to show the current code coverage %.
PR 2373 changed the CI in such a way that coding standards errors will now be shown inline in the code of PRs.
With that change in place, having the information about running PHPCS in the pull request template has become redundant.
Follow up on 2384.
1. This test does not actually need a regular expression to test the mime message, an `assertStringContainsString()` assertion is sufficient.
2. Remove duplicate information from the `$expected` parameter. The `Content-Type: text/calendar; method=` part will always be the same. Only the method name will change.
Including test cases with:
* Spaces in the paths.
* Incomplete file paths.
Note: the "empty string" test may need looking at.... should this return an empty array instead ?
* Replace the existing test code with two new test methods using data providers.
* Maintains the same test code and test cases.
* Makes it easier to add additional test cases in the future.
As this test does not actually need an instantiated PHPMailer object, this class extends the `Yoast\PHPUnitPolyfills\TestCases\TestCase` instead of the `PHPMailer\Test\TestCase`.
Note: this doesn't move the complete test from the original test file, just select parts of the test method.
This commit:
* Add a new dependency on the PHP Parallel Lint package for fast PHP linting.
The PHP Parallel Lint package, in combination with the PHP Console Highlighter provides the following advantages in comparison with "plain" PHP linting:
- Higher speed due to the parallel processes.
- Improved usability by providing color coded syntax highlighting of found errors on the command-line.
- Integration with the `cs2pr` tool, allowing for the results of the lint command to be shown in-line in PRs.
* Adds a Composer `lint` script for easy access to the tool for devs, while making sure the correct command line parameters will be used.
The linting command as currently set up, will also check the example files for linting errors.
* Adds a GH Actions job for linting the code on the high/low supported PHP versions, one arbitrary interim version + an experimental build against PHP 8.1.
The `cs2pr` tool has been enabled and will show the results of the non-experimental lint runs in-line in PRs.
**Note**: For PHP 8.1, the `cs2pr` tool is not used as there is a known issue in the Parallel Lint tool with PHP 8.1 which breaks on the checkstyle reporting. There is already a [PR open](https://github.com/php-parallel-lint/PHP-Parallel-Lint/pull/64) to fix this upstream. Once this PR has been merged and a new version of Parallel Lint has been released, the separate step for PHP 8.1 linting can be removed.
* Makes the `test` job in the GHA workflow dependent on the `lint` job having passed...
... as the tests would fail anyway if there are linting errors.
Also adjusts the name of the `test` jobs to include the word "Test" so they can be easily distinguished from the Lint jobs.
Refs:
* https://github.com/php-parallel-lint/PHP-Parallel-Lint
The `xdebug` extension is already tagged as needed via the `coverage` setting, no need to add it to the `extensions` list.
---
Note: generally speaking, I personally normally don't pass an `extensions` list and allow the `setup-php` action to run with the default extensions, which is sufficient in most cases and would be sufficient here as well.
More than anything, I use the `extensions` key to _disable_ extensions for certain test runs, rather than enable them. Just something to consider.
The below documentation should give more insight.
Refs:
* https://github.com/shivammathur/setup-php/wiki/Php-extensions-loaded-on-ubuntu-18.04
* https://github.com/shivammathur/setup-php#heavy_plus_sign-php-extension-support
It is generally speaking a good idea to cache downloaded Composer packages between runs for performance reasons.
Now, this can be set up manually and would add two more steps to the scripts, or Ben's `composer-install` action can be used which will handle it all for you. The `composer-install` action is versatile and allows for passing additional parameters, so is perfectly suitable for this.
Ref: https://github.com/marketplace/actions/install-composer-dependencies
Running against stable/lowest dependencies is relevant when a package has runtime (non-dev) dependencies.
However, PHPMailer does not have runtime dependencies.
In other words, the `dependency-version` matrix key is redundant and unused, so we may as well remove it.
As this test does not actually need an instantiated PHPMailer object, this class extends the `Yoast\PHPUnitPolyfills\TestCases\TestCase` instead of the `PHPMailer\Test\TestCase`.
Note: this doesn't move the complete test from the original test file, just select parts of the test method.
As this test does not actually need an instantiated PHPMailer object, this class extends the `Yoast\PHPUnitPolyfills\TestCases\TestCase` instead of the `PHPMailer\Test\TestCase`.
Note: this doesn't move the complete tests from the original test file, just select parts of two test methods.
The `PHPMailer::preSend()` method may alter the value of the _`protect static`_ `PHPMailer::$LE` property.
a9978d2079/src/PHPMailer.php (L1494-L1504)
This means that tests which rely on the value of the property being the default, may start to fail unexpectedly depending on the _order_ in which tests are being run.
I.e. if the test relying on the default value is run first, all will be fine, however, if that same test is run _after_ a test which calls `preSend()` on a non-Windows box with PHP < 8.0 and with `$this->Mailer` set to `mail`, the test may fail unexpectedly.
This commit fixes that by expanding the previously introduced "resetting of static properties" logic.
This commit:
* Expands the list of static properties in the `TestCase` class to include the `PHPMailer::$LE` property.
* Introduces two new methods to the `TestCase` class:
- `resetStaticProperties()` to reset all static properties listed in the `TestCase::$PHPMailerStaticProps` property to their default values.
- `updateStaticProperty()` to change the value of one individual static property in a class.
* The `updateStaticProperty()` method was made `public static` to allow for tests which do not need an instantiated PHPMailer object and thus extend the `Yoast\PHPUnitPolyfills\TestCases\TestCase` directly to be able to use the method as well.
If any such tests would use the method, that particular test will be responsible for resetting the value back to the default after the test.
* The resetting of the static properties has also been moved from the `set_up()` to the `tear_down()` method.
As mentioned above, these static properties may be used by both tests extending this `TestCase` as well as test extending the `Yoast\PHPUnitPolyfills\TestCases\TestCase`. With that in mind, resetting _after_ each test will be more stable, as long as tests which extend the `Yoast\PHPUnitPolyfills\TestCases\TestCase` and would change static properties clean up after themselves.
**Note**: I'm not including the `PHPMailer::$IcalMethods` property in this change as AFAICS, this property is not being changed anywhere in the package code and as the property is `protected`, it cannot easily be changed from within a test either (and if it would be changed by a test, that test would be responsible for resetting it).
Also note: due to limitations in the `Reflection` extension, a hard-coded list of the default values of the static properties will need to be maintained in the `TestCase`. This cannot be helped as the `Reflection` extension does not have a way to accurately retrieve the default value of static properties in a PHP cross-version compatible manner.
The `PHPMailer::__construct()` method has an optional `$exceptions` property to throw exceptions on errors.
Up to now, the `PHPMailer` instance created by the `set_up()` would not pass any parameters, effectively instantiating `PHPMailer` without turning on exceptions.
In a test situation it may be useful for tests to test the behaviour of a method _with_ and _without_ the exceptions option and to test that certain exceptions are thrown and throw the correct message.
With that in mind, I'm introducing a `USE_EXCEPTIONS` class constant to the `TestCase` which can be overloaded in individual test classes and will be used by the `set_up()` method to determine whether it will be instantiated with exceptions or not.
Includes introducing an overload of the class constant in one of the test class for which it would seem appropriate at this time.
Note: this does mean that the `testDKIMSigningMail()` test will show as errored instead of failed if no SMTP connection could be made, but that IMO is the more correct status anyhow.
This allows for using the `setAddress()` method in a more consistent manner (where appropriate).
Includes introducing the use of the `setAddress()` function in a few select places.
Note: I do wonder whether this method should ever be used outside of `set_up()` and `tear_down()`, but that is for further discussion and outside the scope of this commit.
While this class does not (currently) _change_ the `PHPMailer::$validator` property, it does rely on the default value being the expected default.
By resetting the value of the property before and after the class, this is safeguarded for the current tests.
Public static properties changed by individual tests were not being reset to their default value prior to the next test being run.
This could influence the test results of subsequent tests as static properties are not automatically reset when a new instance of a class is instantiated. See: https://3v4l.org/8s1RB
Luckily this didn't aversely affect the tests so far, but should be safeguarded for the future.
There are only three static properties in the `PHPMailer` class, with only one of these being `public`.
This commit introduces a code snippet which will reset any static properties as listed in the `$PHPMailerStaticProps` property of the `TestCase` to the default value, as also set in the `$PHPMailerStaticProps` property, at the start of the `set_up()` which is run before each test using the `TestCase`.
For now, this is only relevant for the `ValidateAddressCustomValidatorTest` test and only when the `testSetDefaultValidatorToCustom()` test would fail before resetting the property. All the same, having the reset in place by default will ensure that future tests which change this property won't introduce side-effects to other tests.
... in part to document the behaviour of the function, in part to actually test it.
By the looks of it more tests are still needed though to raise the code coverage of this function to 100%.
I'd also recommend adding tests with different charset settings as the current tests are all based on the default charset (`CHARSET_ISO88591`).
Also note: the "empty string" test may need looking at.... is this a bug ?
* Maintains the existing test cases.
* Prevent one failing assertion hiding a potential second failure.
* Makes it easier to add additional test cases in the future.
As this test is marked _incomplete_, no further review of the test has been done and no `@covers` tag has been added.
Co-authored-by: jrfnl <jrfnl@users.noreply.github.com>
