[11.x] Enhance database migrations

[hafezdivandari]

This PR does a lot of improvements and unlocks many possibilities on database migrations, also makes it easier to maintain!

New Features / Enhancements 🚀

Extend SQLite support to 3.26+

Laravel currently requires SQLite 3.35+, this PR extends SQLite support to 3.26+

Add and Drop Foreign Keys on SQLite

SQLite doesn't support adding / dropping foreign keys with alter table command, but as suggested on the docs, you may recreate the table with arbitrary changes, that's what this PR do to implement these commands.

This PR adds support for blueprint's $table->foreign(), $table->dropForeign() methods and ->constrained() modifier on SQLite:

Schema::table('posts', function (Blueprint $table) {
    $table->unsignedBigInteger('user_id');

    $table->foreign('user_id')->references('id')->on('users'); // Now works as expected on SQLite
});

Schema::table('posts', function (Blueprint $table) {
    $table->foreignId('user_id')
        ->constrained(); // Now works as expected on SQLite
});

Schema::table('posts', function (Blueprint $table) {
    $table->dropForeign(['user_id']); // Now works as expected on SQLite, no exception will be thrown anymore!
});

Fixes #51318, SQLite doesn't allow dropping a column if it is used in a foreign key constraint, this PR makes it possible:

Schema::table('sessions', function (Blueprint $table) {
    $table->dropForeign(['user_id']);
    $table->dropColumn('user_id');
});

// or with more magic...
Schema::table('sessions', function (Blueprint $table) {
    $table->dropConstrainedForeignId('user_id');
});

Add and Drop the primary key on SQLite

This PR adds support for blueprint's $table->primary(), $table->dropPrimary() methods, ->primary(), and ->primary(false)->change() modifiers when modifying table on SQLite:

Schema::table('posts', function (Blueprint $table) {
    $table->dropPrimary();              // Drop the current primary key
    $table->string('email');            // Add 'email' column
    $table->primary(['name', 'email']); // Add a composite primary key 
});

Schema::table('posts', function (Blueprint $table) {
    $table->uuid('id')->primary();  // Add 'id' column and make it primary
});

Schema::table('posts', function (Blueprint $table) {
    $table->string('id')->primary(false)->change();  // Change the type of 'id' column to `string` and drop primary key
});

Preserve the order of commands

This is the most important change on this PR; When updating table and compiling blueprint commands to SQL, the order of commands wasn't respected:

• First compile all modified columns at one query
• Second compile all added columns at once
• Then compile other commands

As discussed on #50925, this will causes issue, unexpected results and leads to error most of the time when updating table. For example, the following code raises The 'username' column already exists! SQL error on all databases, because add column command was compiled before rename column:

Schema::table('users', function (Blueprint $table) {
    $table->renameColumn('username', 'email');
    $table->string('username');
});

This PR fixes this by preserving the order of commands as declared on the blueprint. You don't have to call each command in a separate Schema::table anymore that breaks migration transaction!

Blueprint State for SQLite

As explained before, when updating table on SQLite, we have to recreate the table to do arbitrary changes. To do so we need to know the current state of the table on each command. This PR adds BlueprintState class to keep track of changes on the table. After this PR we are able to mix any combination of blueprint commands in a single Schema::table (as demonstrated on the added integration tests) and it works as expected in the same order!

Notes

1. Unlike other database drivers, foreign keys don't have name on SQLite, so obviously you can't drop a foreign key by its name on SQLite:

   Schema::table('posts', function (Blueprint $table) {
       $table->dropForeign('posts_user_id_foreign'); // ❌ doesn't work on SQLite
   
       $table->dropForeign(['user_id']);             // ✔️ works as expected
   }

2. We probably need to remove withSqlite trait on orchestral/testbench package, it seems we don't need that anymore if it's merged?

[github-actions]

Thanks for submitting a PR!

Note that draft PR's are not reviewed. If you would like a review, please mark your pull request as ready for review in the GitHub user interface.

Pull requests that are abandoned in draft may be closed due to inactivity.

[shaedrich]

Is there a reason, you mixed the SQLite and migration execution order changes into one PR instead of two separate ones?

[ocleroux]

I'm eager to see this merged!

[hafezdivandari]

@shaedrich we can't add SQLite features without fixing the migration commands' orders.

[shaedrich]

@hafezdivandari You could have created the SQLite PR as draft, added a comment concerning the PR merging order and have backmerged the main branch after the migration command order PR was merged. But in the end, it's Taylor's choice to make

[hafezdivandari]

@shaedrich It's fine ;)

[taylorotwell]

To be honest this just is too much for us to take on reviewing and maintaining at the moment and I'm not sure the return on investment is there for end users.

Feels like something maybe better reserved for later this year or early next year for Laravel 12.

[Gandhi11]

@taylorotwell How are you guys dropping columns with foreign key in SQLite currently? I do not understand how this is not a major issue for a lot a Laravel applications.