Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat eldad4 coroutines #7975

Draft
wants to merge 68 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
d8996b4
Implement DB proxy
Meldiron Mar 5, 2024
e807a5d
Upgrade DB proxy
Meldiron Mar 5, 2024
a9731cd
Fix account tests
Meldiron Mar 6, 2024
882d8e5
Fix database tests
Meldiron Mar 6, 2024
f83d6e6
PR review changes
Meldiron Mar 6, 2024
4fa9ccd
upgrade libs
Meldiron Mar 6, 2024
1b65c5f
linter fix
Meldiron Mar 6, 2024
27e995e
Reomve leftpover
Meldiron Mar 6, 2024
ee065bc
Merge branch '1.5.x' into feat-database-proxy
Meldiron Mar 7, 2024
8a8638a
Formatting fix
Meldiron Mar 7, 2024
5a7c43a
Introduce code analysis
Meldiron Mar 7, 2024
efeb898
Import fixes
Meldiron Mar 7, 2024
005a239
Auth fixes
Meldiron Mar 7, 2024
db16748
Finish fixing code QL warnings
Meldiron Mar 8, 2024
d5f180d
Add debugging
Meldiron Mar 8, 2024
513bf7e
Update .env
Meldiron Mar 8, 2024
4de9591
Disable debug
Meldiron Mar 8, 2024
2a0a69f
Fix connections pool
Meldiron Mar 9, 2024
713928c
Couroutines test
eldadfux Apr 1, 2024
6ce8781
Fixed email escaping
eldadfux Apr 1, 2024
c73ef2c
Fixed 451 status code
eldadfux Apr 1, 2024
b570368
Fixed auth error in users usage
eldadfux Apr 1, 2024
5ce3de4
Merge remote-tracking branch 'origin/main' into feat-eldad2-coroutines
eldadfux Apr 2, 2024
be7a6e5
Fixed whitespace
eldadfux Apr 2, 2024
378bb5f
Fixed server
eldadfux Apr 2, 2024
a1db69b
Fixed missing System namespace
eldadfux Apr 2, 2024
bd57955
Enabled coroutines
eldadfux Apr 3, 2024
211bcf8
Removed proxy db
eldadfux Apr 9, 2024
08b3182
WIP
eldadfux Apr 14, 2024
9e234b7
updated server
eldadfux Apr 14, 2024
6cade89
New init structure
eldadfux Apr 14, 2024
766b2ba
Avatars tests are green!
eldadfux Apr 14, 2024
fd2410d
Fixed bootstrap
eldadfux Apr 14, 2024
a5ba262
Fixed bootstrap file
eldadfux Apr 15, 2024
49a50b3
Test DB failure
eldadfux Apr 15, 2024
87ca2df
Applied fixes for tests
eldadfux Apr 15, 2024
13eb3bc
Fixed users tests
eldadfux Apr 15, 2024
07a830e
Work on the main worker
eldadfux Apr 15, 2024
2e4c6ea
Merge branch 'main' into feat-eldad2-coroutines
eldadfux Apr 15, 2024
6d6f227
Eanble compression
eldadfux Apr 15, 2024
479f20d
Fixed some workers
eldadfux Apr 22, 2024
74dc995
Fixed console test
eldadfux Apr 22, 2024
aff328c
Fixed test
eldadfux Apr 22, 2024
b97cf78
Enabled more controllers
eldadfux Apr 22, 2024
3b70ae4
Fixed formatting
eldadfux Apr 22, 2024
c413a6c
Updated database service
eldadfux Apr 22, 2024
5d54f56
Fixed client overwriting
eldadfux Apr 22, 2024
ab6744d
Fixed webhooks tests
eldadfux Apr 22, 2024
19f8b13
Fixed formatting
eldadfux Apr 22, 2024
6a67a1d
Fixed execute to match new signature
eldadfux Apr 22, 2024
351b931
Fixed router, and options
eldadfux Apr 22, 2024
09e9483
Changed autoloading order
eldadfux Apr 22, 2024
99994f6
Fixed storage tests
eldadfux Apr 22, 2024
86b4635
Fixed for general tests
eldadfux Apr 22, 2024
6c8b228
Merge branch 'main' into feat-eldad4-coroutines
eldadfux Apr 22, 2024
d67df5f
Fixed formatting
eldadfux Apr 22, 2024
7381aba
Merge remote-tracking branch 'refs/remotes/origin/feat-eldad4-corouti…
eldadfux Apr 22, 2024
69a43aa
Fixed projects tests
eldadfux Apr 22, 2024
46ab7b1
Fixed functions test
eldadfux Apr 22, 2024
beeb66e
Fixed account tests
eldadfux Apr 22, 2024
9fc0342
Updated injections
eldadfux Apr 23, 2024
55fe725
Fixed format
eldadfux Apr 24, 2024
90ef64c
Fixed namespace
eldadfux May 8, 2024
c53de9f
Merge branch 'main' of github.com:appwrite/appwrite into feat-eldad4-…
eldadfux May 8, 2024
5cfe4db
Updated console
eldadfux May 8, 2024
0b7c795
Updated lock file
eldadfux May 8, 2024
525383c
Updates to realtime
eldadfux May 8, 2024
90fd0ba
Updated db lib
eldadfux May 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
16 changes: 16 additions & 0 deletions .github/workflows/codeql-phpstan.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: "CodeQL"

on: [pull_request]
jobs:
lint:
name: CodeQL
runs-on: ubuntu-latest

steps:
- name: Check out the repo
uses: actions/checkout@v2

- name: Run CodeQL
run: |
docker run --rm -v $PWD:/app composer sh -c \
"composer install --profile --ignore-platform-reqs && composer check"
2 changes: 1 addition & 1 deletion CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -434,7 +434,7 @@
* Bump console to version 3.2.7 [#7148](https://github.com/appwrite/appwrite/pull/7148)
* Chore update database to 0.45.2 [#7138](https://github.com/appwrite/appwrite/pull/7138)
* Implement queue thresholds for the health API [#7123](https://github.com/appwrite/appwrite/pull/7123)
* Add Authorization::skip to the usage worker [#7124](https://github.com/appwrite/appwrite/pull/7124)
* Add $auth->skip to the usage worker [#7124](https://github.com/appwrite/appwrite/pull/7124)

## Bug fixes
* fix: use queueForDeletes in git installation delete endpoint [#7140](https://github.com/appwrite/appwrite/pull/7140)
Expand Down
76 changes: 50 additions & 26 deletions app/cli.php
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
<?php

require_once __DIR__ . '/init.php';
require_once __DIR__ . '/init2.php';
require_once __DIR__ . '/controllers/general.php';

use Appwrite\Event\Certificate;
use Appwrite\Event\Delete;
use Appwrite\Event\Func;
use Appwrite\Event\Hamster;
use Appwrite\Platform\Appwrite;
use Appwrite\Utopia\Queue\Connections;
use Utopia\Cache\Adapter\Sharding;
use Utopia\Cache\Cache;
use Utopia\CLI\CLI;
Expand All @@ -23,45 +24,48 @@
use Utopia\Registry\Registry;
use Utopia\System\System;

Authorization::disable();
global $register;

CLI::setResource('register', fn () => $register);

CLI::setResource('cache', function ($pools) {
CLI::setResource('connections', function () {
return new Connections();
});

CLI::setResource('cache', function ($pools, Connections $connections) {
$list = Config::getParam('pools-cache', []);
$adapters = [];

foreach ($list as $value) {
$adapters[] = $pools
->get($value)
->pop()
->getResource()
;
$connection = $pools->get($value)->pop();
$connections->add($connection);
$adapters[] = $connection->getResource();
}

return new Cache(new Sharding($adapters));
}, ['pools']);
}, ['pools', 'connections']);

CLI::setResource('pools', function (Registry $register) {
return $register->get('pools');
}, ['register']);

CLI::setResource('dbForConsole', function ($pools, $cache) {
CLI::setResource('dbForConsole', function ($pools, $cache, $auth, Connections $connections) {
$sleep = 3;
$maxAttempts = 5;
$attempts = 0;
$ready = false;

$connection = null;

do {
$attempts++;
try {
// Prepare database connection
$dbAdapter = $pools
->get('console')
->pop()
->getResource();
$connection = $pools->get('console')->pop();
$dbAdapter = $connection->getResource();

$dbForConsole = new Database($dbAdapter, $cache);
$dbForConsole->setAuthorization($auth);

$dbForConsole
->setNamespace('_console')
Expand All @@ -78,23 +82,31 @@

$ready = true;
} catch (\Throwable $err) {
if($connection !== null) {
$connection->reclaim();
$connection = null;
}

Console::warning($err->getMessage());
$pools->get('console')->reclaim();
sleep($sleep);
}
} while ($attempts < $maxAttempts && !$ready);

if($connection !== null) {
$connections->add($connection);
}

if (!$ready) {
throw new Exception("Console is not ready yet. Please try again later.");
}

return $dbForConsole;
}, ['pools', 'cache']);
}, ['pools', 'cache', 'auth', 'connections']);

CLI::setResource('getProjectDB', function (Group $pools, Database $dbForConsole, $cache) {
CLI::setResource('getProjectDB', function (Group $pools, Database $dbForConsole, $cache, $auth, Connections $connections) {
$databases = []; // TODO: @Meldiron This should probably be responsibility of utopia-php/pools

return function (Document $project) use ($pools, $dbForConsole, $cache, &$databases) {
return function (Document $project) use ($pools, $dbForConsole, $cache, &$databases, $auth, $connections) {
if ($project->isEmpty() || $project->getId() === 'console') {
return $dbForConsole;
}
Expand All @@ -107,12 +119,12 @@
return $database;
}

$dbAdapter = $pools
->get($databaseName)
->pop()
->getResource();
$connection = $pools->get($databaseName)->pop();
$connections->add($connection);
$dbAdapter = $connection->getResource();

$database = new Database($dbAdapter, $cache);
$database->setAuthorization($auth);

$databases[$databaseName] = $database;

Expand All @@ -123,11 +135,13 @@

return $database;
};
}, ['pools', 'dbForConsole', 'cache']);
}, ['pools', 'dbForConsole', 'cache', 'auth', 'connections']);

CLI::setResource('queue', function (Group $pools) {
return $pools->get('queue')->pop()->getResource();
}, ['pools']);
CLI::setResource('queue', function (Group $pools, Connections $connections) {
$connection = $pools->get('queue')->pop();
$connections->add($connection);
return $connection->getResource();
}, ['pools', 'connections']);
CLI::setResource('queueForFunctions', function (Connection $queue) {
return new Func($queue);
}, ['queue']);
Expand Down Expand Up @@ -165,6 +179,7 @@
$log->setAction($action);

$isProduction = System::getEnv('_APP_ENV', 'development') === 'production';

$log->setEnvironment($isProduction ? Log::ENVIRONMENT_PRODUCTION : Log::ENVIRONMENT_STAGING);

$responseCode = $logger->addLog($log);
Expand All @@ -176,11 +191,20 @@
};
}, ['register']);

CLI::setResource('auth', fn () => new Authorization());

$platform = new Appwrite();
$platform->init(Service::TYPE_CLI);

$cli = $platform->getCli();

$cli
->init()
->inject('auth')
->action(function (Authorization $auth) {
$auth->disable();
});

$cli
->error()
->inject('error')
Expand Down
2 changes: 1 addition & 1 deletion app/console
Submodule console updated 39 files
+15 −9 package-lock.json
+3 −2 package.json
+0 −5 src/lib/actions/analytics.ts
+31 −43 src/lib/components/copyInput.svelte
+0 −1 src/lib/components/drop.svelte
+1 −5 src/lib/components/heading.svelte
+5 −7 src/lib/components/modal.svelte
+5 −35 src/lib/elements/forms/inputDigits.svelte
+11 −23 src/lib/elements/forms/inputFile.svelte
+5 −33 src/lib/layout/unauthenticated.svelte
+0 −1 src/lib/stores/billing.ts
+6 −10 src/lib/stores/stripe.ts
+2 −2 src/lib/wizards/functions/components/repositories.svelte
+14 −2 src/routes/+layout.svelte
+6 −0 src/routes/+layout.ts
+3 −5 src/routes/console/+error.svelte
+5 −7 src/routes/console/+layout.svelte
+7 −3 src/routes/console/account/deleteMfa.svelte
+105 −23 src/routes/console/account/mfa.svelte
+0 −64 src/routes/console/account/mfaRecoveryCodes.svelte
+0 −48 src/routes/console/account/mfaRegenerateCodes.svelte
+0 −11 src/routes/console/account/payments/+page.svelte
+1 −2 src/routes/console/account/payments/paymentModal.svelte
+1 −4 src/routes/console/account/store.ts
+55 −173 src/routes/console/account/updateMfa.svelte
+1 −1 src/routes/console/organization-[organization]/billing/replaceCard.svelte
+1 −1 src/routes/console/organization-[organization]/billing/retryPaymentModal.svelte
+1 −2 src/routes/console/organization-[organization]/billing/wizard/paymentDetails.svelte
+14 −25 src/routes/console/organization-[organization]/header.svelte
+3 −3 src/routes/console/project-[project]/auth/user-[user]/updateMfa.svelte
+5 −1 src/routes/console/project-[project]/overview/platforms/createAndroid.svelte
+5 −2 src/routes/console/project-[project]/overview/platforms/createApple.svelte
+5 −2 src/routes/console/project-[project]/overview/platforms/createFlutter.svelte
+5 −2 src/routes/console/project-[project]/overview/platforms/createWeb.svelte
+9 −4 src/routes/console/wizard/cloudOrganizationChangeTier/paymentDetails.svelte
+9 −152 src/routes/mfa/+page.svelte
+0 −8 src/routes/mfa/+page.ts
+ static/images/vcs/status-building-dark.gif
+ static/images/vcs/status-building-light.gif