Server is not reachable on iOS

[d4nnyx]

Summary

When CORS is disabled on a server and nginx does not pass the origin header through proxy, iOS shows banner "Server is not reachable", but everything seems working.

Environment Information

• Device Name: iPhone 14 Pro
• OS Version: iOS 17.2.1
• Mattermost App Version: v2.12.0
• Mattermost Server Version: 8.1.9 (self-hosted)

Steps to reproduce

1. Set nginx as a proxy with provided configuration
2. Disable CORS in System Console

Expected behavior

No error banner, same as in the desktop or web app

Observed behavior (that appears unintentional)

iOS logs

websocket error wss://<host>/api/v4/websocket?connection_id=&sequence_number=0
Handling Javascript error WEBSOCKET ERROR EVENT 0
WEBSOCKET ERROR EVENT {
    message =     {
        error =         {
            message = "Starscream.HTTPUpgradeError.notAnUpgrade(403, [\"Strict-Transport-Security\": \"max-age=31536000; includeSubDomains\", \"Sec-WebSocket-Version\": \"13\", \"X-Frame-Options\": \"SAMEORIGIN\", \"Vary\": \"Accept-Encoding\", \"X-Request-Id\": \"m9ebrrbk6pndmg1prjs7icd9dy\", \"X-XSS-Protection\": \"1; mode=block\", \"Server\": \"nginx\", \"Content-Length\": \"191\", \"Expires\": \"0\", \"Date\": \"Fri, 19 Jan 2024 23:47:16 GMT\", \"X-Content-Type-Options\": \"nosniff, nosniff\", \"Connection\": \"keep-alive\", \"Permissions-Policy\": \"\", \"Referrer-Policy\": \"no-referrer\", \"Content-Type\": \"text/plain; charset=utf-8\", \"X-Version-Id\": \"8.1.9.7509144995.43ad6fe6c0219880ef6f11aeea5e7859.false\"])";
        };
    };
    url = "wss://<host>/api/v4/websocket?connection_id=&sequence_number=0";
}
Handling Javascript error websocket error 0
websocket error wss://<host>/api/v4/websocket?connection_id=&sequence_number=0
Handling Javascript error WEBSOCKET ERROR EVENT 0
WEBSOCKET ERROR EVENT {
    message =     {
        error =         {
            message = "Starscream.WSError(type: Starscream.ErrorType.protocolError, message: \"masked and rsv data is not currently supported\", code: 1002)";
        };
    };
    url = "wss://<host>/api/v4/websocket?connection_id=&sequence_number=0";
}
WEBSOCKET RECONNECT MODELS BATCHING TOOK 8ms

Mattermost server logs

{"timestamp":"2024-01-20 09:26:12.039 Z","level":"debug","msg":"Failed to upgrade websocket connection.","caller":"web/context.go:113","path":"/api/v4/websocket","request_id":"3nxe19op8jb9ubn9jteca9b3ga","ip_addr":"127.0.0.1","user_id":"adwp74goufycfru1p8m4ptra5w","method":"GET","err_where":"connect","http_code":400,"error":"connect: Failed to upgrade websocket connection., websocket: request origin not allowed by Upgrader.CheckOrigin"}

NGINX server logs

37.235.104.12 - - [20/Jan/2024:09:27:11 +0000] "GET /api/v4/websocket?connection_id=wp9t4o6zj389ig3h6d518sbn4e&sequence_number=1 HTTP/1.0" 403 191 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.5993.89 Electron/27.0.2 Safari/537.36 Mattermost/5.6.0" "<ip>"

NGINX configuration

server {
    server_name _;
    listen 80 default;

  	location = /robots.txt {
        add_header Content-Type text/plain;
        return 200 "User-agent: *\nDisallow: /\n";
    }

    location ~ /api/v[0-9]+/(users/)?websocket$ {
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        client_max_body_size 50M;
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Frame-Options SAMEORIGIN;
        proxy_set_header Early-Data $ssl_early_data;
        proxy_buffers 256 16k;
        proxy_buffer_size 16k;
        client_body_timeout 60;
        send_timeout 300;
        lingering_timeout 5;
        proxy_connect_timeout 90;
        proxy_send_timeout 300;
        proxy_read_timeout 90s;
        proxy_http_version 1.1;
        proxy_pass http://localhost:8065;
    }

    location ~ /api/v[0-9]+/plugins$ {
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        client_max_body_size 100M;
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Frame-Options SAMEORIGIN;
        proxy_set_header Early-Data $ssl_early_data;
        proxy_buffers 256 16k;
        proxy_buffer_size 16k;
        client_body_timeout 60;
        send_timeout 300;
        lingering_timeout 5;
        proxy_connect_timeout 90;
        proxy_send_timeout 300;
        proxy_read_timeout 90s;
        proxy_http_version 1.1;
        proxy_pass http://localhost:8065;
    }

  	location / {
        client_max_body_size 50M;
        proxy_set_header Connection "";
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Frame-Options SAMEORIGIN;
        proxy_set_header Early-Data $ssl_early_data;
        proxy_buffers 256 16k;
        proxy_buffer_size 16k;
        proxy_read_timeout 600s;
        proxy_cache mattermost_cache;
        proxy_cache_revalidate on;
        proxy_cache_min_uses 2;
        proxy_cache_use_stale timeout;
        proxy_cache_lock on;
        proxy_http_version 1.1;
        proxy_pass http://localhost:8065;
    }
}

Mattermost config

"ServiceSettings": {
        "SiteURL": "",
        "WebsocketURL": "",
        "LicenseFileLocation": "",
        "ListenAddress": ":8065",
        "ConnectionSecurity": "",
        "TLSCertFile": "",
        "TLSKeyFile": "",
        "TLSMinVer": "1.2",
        "TLSStrictTransport": false,
        "TLSStrictTransportMaxAge": 63072000,
        "TLSOverwriteCiphers": [],
        "UseLetsEncrypt": false,
        "LetsEncryptCertificateCacheFile": "./config/letsencrypt.cache",
        "Forward80To443": false,
        "TrustedProxyIPHeader": [],
        "ReadTimeout": 300,
        "WriteTimeout": 300,
        "IdleTimeout": 60,
        "MaximumLoginAttempts": 10,
        "GoroutineHealthThreshold": -1,
        "EnableOAuthServiceProvider": true,
        "EnableIncomingWebhooks": true,
        "EnableOutgoingWebhooks": true,
        "EnableCommands": true,
        "EnablePostUsernameOverride": false,
        "EnablePostIconOverride": false,
        "GoogleDeveloperKey": "",
        "EnableLinkPreviews": true,
        "EnablePermalinkPreviews": true,
        "RestrictLinkPreviews": "",
        "EnableTesting": false,
        "EnableDeveloper": false,
        "DeveloperFlags": "",
        "EnableClientPerformanceDebugging": false,
        "EnableOpenTracing": false,
        "EnableSecurityFixAlert": true,
        "EnableInsecureOutgoingConnections": false,
        "AllowedUntrustedInternalConnections": "",
        "EnableMultifactorAuthentication": false,
        "EnforceMultifactorAuthentication": false,
        "EnableUserAccessTokens": false,
        "AllowCorsFrom": "",
        "CorsExposedHeaders": "",
        "CorsAllowCredentials": false,
        "CorsDebug": false,
        "AllowCookiesForSubdomains": false,
        "ExtendSessionLengthWithActivity": false,
        "SessionLengthWebInDays": 180,
        "SessionLengthWebInHours": 4320,
        "SessionLengthMobileInDays": 180,
        "SessionLengthMobileInHours": 4320,
        "SessionLengthSSOInDays": 30,
        "SessionLengthSSOInHours": 720,
        "SessionCacheInMinutes": 10,
        "SessionIdleTimeoutInMinutes": 43200,
        "WebsocketSecurePort": 443,
        "WebsocketPort": 80,
        "WebserverMode": "gzip",
        "EnableGifPicker": true,
        "GfycatAPIKey": "2_KtH_W5",
        "GfycatAPISecret": "3wLVZPiswc3DnaiaFoLkDvB4X0IV6CpMkj4tf2inJRsBY6-FnkT08zGmppWFgeof",
        "GiphySdkKey": "",
        "EnableCustomEmoji": true,
        "EnableEmojiPicker": true,
        "PostEditTimeLimit": -1,
        "TimeBetweenUserTypingUpdatesMilliseconds": 5000,
        "EnablePostSearch": true,
        "EnableFileSearch": true,
        "MinimumHashtagLength": 3,
        "EnableUserTypingMessages": true,
        "EnableChannelViewedMessages": true,
        "EnableUserStatuses": true,
        "ExperimentalEnableAuthenticationTransfer": true,
        "ClusterLogTimeoutMilliseconds": 2000,
        "EnablePreviewFeatures": true,
        "EnableTutorial": true,
        "EnableOnboardingFlow": true,
        "ExperimentalEnableDefaultChannelLeaveJoinMessages": true,
        "ExperimentalGroupUnreadChannels": "disabled",
        "EnableAPITeamDeletion": false,
        "EnableAPITriggerAdminNotifications": false,
        "EnableAPIUserDeletion": false,
        "ExperimentalEnableHardenedMode": false,
        "ExperimentalStrictCSRFEnforcement": false,
        "EnableEmailInvitations": true,
        "DisableBotsWhenOwnerIsDeactivated": true,
        "EnableBotAccountCreation": true,
        "EnableSVGs": true,
        "EnableLatex": true,
        "EnableInlineLatex": true,
        "PostPriority": true,
        "AllowPersistentNotifications": true,
        "AllowPersistentNotificationsForGuests": false,
        "PersistentNotificationIntervalMinutes": 5,
        "PersistentNotificationMaxCount": 6,
        "PersistentNotificationMaxRecipients": 5,
        "EnableAPIChannelDeletion": false,
        "EnableLocalMode": false,
        "LocalModeSocketLocation": "/var/tmp/mattermost_local.socket",
        "EnableAWSMetering": false,
        "SplitKey": "",
        "FeatureFlagSyncIntervalSeconds": 30,
        "DebugSplit": false,
        "ThreadAutoFollow": true,
        "CollapsedThreads": "always_on",
        "ManagedResourcePaths": "",
        "EnableCustomGroups": true,
        "SelfHostedPurchase": true,
        "AllowSyncedDrafts": true,
        "UniqueEmojiReactionLimitPerPost": 50
    },

Unfortunately, I hadn't chance to try it on Android device.

Possible fixes

It is probably caused by badly configured passing of the Origin header through proxy, but I think that all platform should treat this the same way. And in this case, only iOS shows the problem.
After setting allowCorsFrom: '*', banner dissapears.

[amyblais]

Hi @d4nnyx, would you be open to reading through the suggestions and questions in this other GitHub issue to see if they are helpful #7739?

• Mattermost version 2.11.0 reporting bugs in the mobile app ever since the upgrade.  #7739 (comment)
• Mattermost version 2.11.0 reporting bugs in the mobile app ever since the upgrade.  #7739 (comment)
• Mattermost version 2.11.0 reporting bugs in the mobile app ever since the upgrade.  #7739 (comment)

[mjun0812]

I'm in a similar situation and having the same problem.
This problem occurs only with the iOS/Android client and not with the Desktop Client or browser.
This problem does not occur if allowCorsFrom: '*' is set.

[amyblais]

Opened https://mattermost.atlassian.net/browse/MM-56753.

[TLmawu]

We're experiencing the the same behaviour but only on iOS mobile devices. The problem occurred after updating the mattermost app to 2.12.
The "workaround" allowCorsFrom: '*' doesn't help. Mattermost is served by a KEMP LB without Nginx.

Mattermost 8.1.11 ESR
iPhone 13
iOS 17.4
Mattermost Mobile 2.14