Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Array formulae in the Calculation Engine #2787

Draft
wants to merge 145 commits into
base: master
Choose a base branch
from
Draft

Conversation

MarkBaker
Copy link
Member

@MarkBaker MarkBaker commented Apr 28, 2022

This is:

- [ ] a bugfix
- [ ] a new feature
- [ ] refactoring
- [ ] additional unit tests

Checklist:

  • Changes are covered by unit tests
    • Changes are covered by existing unit tests
    • New unit tests have been added
  • Code style is respected
  • Commit message explains why the change is made (see https://github.com/erlang/otp/wiki/Writing-good-commit-messages)
  • CHANGELOG.md contains a short summary of the change
  • Documentation is updated as necessary

Why this change is needed?

Creation of the version 2.0 development branch.

All changes merged into master should also be merged here to keep it up-to-date with bugfixes and any new features introduced in master

It should not be merged to master. When the time is right, we'll have 1.x and 2.x branches, this forming the basis of the 2.x branch

MarkBaker added 30 commits January 30, 2022 19:17
Only supports the basic happy path (no handling for Error results when reading)

No functionality yet for setting array formulae in code
… spillage cells when calculating the value

No provision yet for the single operator, or for handlng #SPILL! or #CALC! errors
…on is written and linked; and then tag array formula cells with the `cm` tag to link them to that cell metadata definition
…mula range dimensions, expanding or shrinking it as necessary.
Note that the deprecated Excel function implementation classes should be removed completely for PhpSpreadsheet 2.0.0; and the bc break that array formula handling entails will require this to be a 2.0.0 implementation... but just for the moment, this is a cleanup... those files will be deleted once we switch to development on a 2.0-dev branch, and I want to do a few more PR merges before then, and update this branch as part of the 2.0-dev changes
# Conflicts:
#	src/PhpSpreadsheet/Calculation/MathTrig.php
#	src/PhpSpreadsheet/Calculation/MathTrig/Absolute.php
#	tests/PhpSpreadsheetTests/Calculation/Functions/MathTrig/AbsTest.php
# Conflicts:
#	src/PhpSpreadsheet/Cell/Cell.php
#	src/PhpSpreadsheet/Writer/Xlsx/Worksheet.php
@oleibman
Copy link
Collaborator

@MarkBaker I have a security-related question that I would like to discuss with you privately. I sent an email to an address I had used to communicate with you about another security-related issue in March, but a new message to that address bounced yesterday. How can we discuss it?

# Conflicts:
#	docs/topics/calculation-engine.md
#	phpstan-baseline.neon
#	src/PhpSpreadsheet/Reader/Xlsx.php
#	tests/PhpSpreadsheetTests/Calculation/Functions/Engineering/ConvertUoMTest.php
#	tests/PhpSpreadsheetTests/Calculation/Functions/Engineering/ErfCTest.php
#	tests/PhpSpreadsheetTests/Calculation/Functions/Engineering/ErfPreciseTest.php
#	tests/PhpSpreadsheetTests/Calculation/Functions/Engineering/ErfTest.php
#	tests/PhpSpreadsheetTests/Calculation/Functions/Engineering/ParseComplexTest.php
#	tests/PhpSpreadsheetTests/Calculation/Functions/Financial/IrrTest.php
#	tests/PhpSpreadsheetTests/Calculation/Functions/LookupRef/AddressTest.php
#	tests/PhpSpreadsheetTests/Calculation/Functions/LookupRef/ColumnsTest.php
#	tests/PhpSpreadsheetTests/Calculation/Functions/LookupRef/IndexTest.php
#	tests/PhpSpreadsheetTests/Calculation/Functions/LookupRef/IndirectTest.php
#	tests/PhpSpreadsheetTests/Calculation/Functions/LookupRef/RowsTest.php
#	tests/PhpSpreadsheetTests/Calculation/Functions/MathTrig/FactTest.php
#	tests/PhpSpreadsheetTests/Calculation/Functions/TextData/NumberValueTest.php
@MarkBaker
Copy link
Member Author

Screaming into the void

image

I really do not like the way the tests have been rewritten; with an abstract base method being used to handle all the test calls and assertions; it hides everything from the actual unit test code.

It was a nightmare to merge so many files manually; and it's created a new nightmare that will take hours and hours of work to fix

@oleibman
Copy link
Collaborator

oleibman commented Dec 7, 2022

I'm sure that must have been my doing, so please accept my apologies. Responding to a user issue, I found there was a huge number of deprecations that had previously been unreported because of lower/uppercase considerations. I got rid of most of them (now and for future module moves) by changing the tests to run in spreadsheet context rather than by direct calls, and, since the logic was basically the same for everything (populate the sheet then calculate the result), it simplified matters a lot to use a common routine that all the tests called. I am sorry that this caused problems for you which I did not anticipate. Is there anything I can do to help?

@PowerKiKi PowerKiKi marked this pull request as draft May 31, 2023 02:35
@PowerKiKi
Copy link
Member

@MarkBaker it seems this branch went stale. Are you still planning to work on it in the future ? or should we close the PR ?

@MarkBaker
Copy link
Member Author

There's some major changes here for handling array formulae in the Calculation Engine. Unfortunately, too many changes to master made it almost impossible to keep it up-to-date. That feature did include a bc break for anybody using custom read filters, hence the v2.0 naming.

I do want to implement those array formulae changes at some point, I'll need to extract them from the branch and re-implement against a new branch. Hopefully, I'll find more time to work on it this year after the craziness of the last year. So I'd prefer to keep the branch for the moment. Once I'm employed again, I'll take another look at which commits I can take across to a new version 2 branch

@PowerKiKi
Copy link
Member

Fair enough. As a head up I'd like to remind you of #3788 (comment) where I mention that I planned to release a v2 because of added native typing pretty much everywhere.

Is that OK with you ? Would you have any breaking changes you want to add before releasing (hopefully in a few days) ?

@MarkBaker
Copy link
Member Author

Fair enough. As a head up I'd like to remind you of #3788 (comment) where I mention that I planned to release a v2 because of added native typing pretty much everywhere.

Is that OK with you ? Would you have any breaking changes you want to add before releasing (hopefully in a few days) ?

I'd go ahead with the deployment of v2.0. I have some new documentation to add before then (I'll try to do those this week), but I won't be able to extract my array formulae changes in the next few days (too much job hunting and preparing for my Netherlands naturalisation exams that take priority) so that will wait for a v3.0, perhaps in a few weeks time.

@PowerKiKi PowerKiKi changed the title 2.0 development Array formulae in the Calculation Engine Jan 24, 2024
@PowerKiKi
Copy link
Member

@MarkBaker, @oleibman, I just released https://github.com/PHPOffice/PhpSpreadsheet/releases/tag/2.0.0 🎉

@oleibman
Copy link
Collaborator

@PowerKiKi Wonderful! Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

None yet

3 participants