Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consolidate content versioning #22413

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open

Conversation

licitdev
Copy link
Member

@licitdev licitdev commented May 7, 2024

Scope

What's changed:

  • Added a new delta field to the directus_versions table
    • Contains the final delta which is the combination of all revisions

Potential Risks / Drawbacks

  • Updates to existing revisions during the migration may result in an incorrect final delta stored
    • This happens when an existing instance of Directus is performing the patch
    • The final delta could be fully repopulated in a subsequent release

Review Notes / Questions

  • The existing versions field in the directus_revisions table will be removed in a subsequent release
  • If the delta field in directus_versions is empty, the existing behaviour of fetching from directus_revisions remains
  • Needs further testing to ensure content versioning remains fully operational

Related to #17894
PR carried over from #22227 which was reverted in #22412.

Copy link

changeset-bot bot commented May 7, 2024

🦋 Changeset detected

Latest commit: c0ad03f

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 21 packages
Name Type
@directus/system-data Patch
@directus/types Patch
@directus/api Patch
@directus/sdk Patch
@directus/utils Patch
@directus/composables Patch
@directus/extensions-sdk Patch
@directus/extensions Patch
@directus/themes Patch
@directus/validation Patch
directus Patch
@directus/env Patch
@directus/extensions-registry Patch
@directus/memory Patch
@directus/pressure Patch
@directus/storage-driver-azure Patch
@directus/storage-driver-cloudinary Patch
@directus/storage-driver-gcs Patch
@directus/storage-driver-s3 Patch
@directus/storage-driver-supabase Patch
create-directus-extension Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@br41nslug br41nslug assigned br41nslug and unassigned paescuj May 7, 2024
@br41nslug

This comment was marked as resolved.

Copy link
Member

@paescuj paescuj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


const finalVersionDelta = assign({}, existingDelta, revisionDelta ? JSON.parse(revisionDelta) : null);

await this.updateOne(key, { delta: finalVersionDelta });
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Saving a new version now requires update permission on directus_versions due to this call - we may want to disregard permissions here (need to check whether that is safe), make it a breaking change requiring update permissions, or find another solution

@@ -167,10 +171,11 @@ export class VersionsService extends ItemsService {
}

override async updateMany(keys: PrimaryKey[], data: Partial<Item>, opts?: MutationOptions): Promise<PrimaryKey[]> {
// Only allow updates on "key" and "name" fields
// Only allow updates on "key", "name" and "delta" fields
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should handle delta updates inside the save method and don't allow direct updates, instead keeping updateMany for updates to key, name only.

Currently, would also do unnecessary duplicate work when called via save, fetching and merging getVersionSavesById two times.

If we should keep it here, reference docs need to be updated 1 where we would need to make clear what the differences are.

Footnotes

  1. https://docs.directus.io/reference/system/versions.html#update-multiple-content-versions

Comment on lines +127 to 130
return [versions[0]['delta']];
}

const saves = await this.getVersionSavesById(versions[0]['id']);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels strange, that this method now (in most/all cases?) returns only one item (the delta), especially while it is called getVersionSaves.
Do we even need that method and getVersionSavesById anymore? Can't we just always use the delta? I think the associated revision items now only serve the purpose of history, but not as source of truth anymore.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This getVersionSavesById is just for the edge case where a new version is created but on the older system when the migration is being run. All usages of it are part of the else clause where the delta is missing. 🤔

Copy link
Member

@paescuj paescuj May 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we should/can account for this edge case. I think this would mean we could never really get rid of it. In any case, I think this method needs some clean-up, for the first reason stated above.

Comment on lines 12 to 15
const deltas = sortBy(
await knex.select('id', 'delta').from('directus_revisions').where('version', '=', version.id),
'id',
).map((revision) => (typeof revision.delta === 'string' ? JSON.parse(revision.delta) : revision.delta));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can/should sort directly via DB query

@@ -238,9 +244,9 @@ router.post(

await service.save(req.params['pk']!, req.body);

const saves = await service.getVersionSavesById(req.params['pk']!);
const updatedVersion = await service.readOne(req.params['pk']!);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could update save to return the delta and use that instead of an additional readOne call.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 📋 Backlog
Development

Successfully merging this pull request may close these issues.

None yet

3 participants