Fix hardware keyboard detection in Shortcuts Bar setup

[l2dy]

• Update isHardwareKB in KBTracker when keyboard state changes.
• Stop overriding inputAssistantItem, which is advised against in UIKit's documentation.
• Add comment on why _keyboardWillChangeFrame is disabled.

In 9e5e024, _keyboardWillChangeFrame is disabled with an early return, therefore detectHardwareKBWithSoftwareKBHeight is not called to update keyboard state.

[carloscabanero]

Hi! Thanks for the fix. Can you tell us, functionality wise, what is the issue you were experiencing? I know the early return is weird but I think this responded to some changes on iOS16 (and we may have left that there for reference, although it should be commented).

[l2dy]

Sure! At first, I was trying to run Blink on an iPad and noticed that the smart keys bar was missing. I then tried it on a simulator with hardware keyboard disconnected and reproduced the same behavior.

With KBTracker.shared.isHardwareKB replaced by isHardwareKB, I was able to get smart keys bar to show up, but when I connect the hardware keyboard to the simulator, the smart keys still exist and overlaps with the iPadOS system shortcuts.

I then found out that isHardwareKB is not updated anymore with the early return. I tried to remove it, and that fixed the bug for me.

At this time, I also realized that the first change was just doing code clean up and did not fix the actual issue of keyboard state management. When there are multiple instances of SmarterTermInput: KBWebView, background webviews may have stale kbView.traits until it becomeFirstResponder(), which calls sync() to update kbView.traits, so there is a subtle difference.

System Information

OS version: iPadOS 17.1.1
Hardware/Simulator: iPad (10th generation)
Blink version: commit cb37c2e

[l2dy]

I tested it on the TestFlight version, and could not reproduce this issue.

Could it be the entitlements I removed that caused the issue? I don't have a paid developer account, so I had to remove these:

diff --git a/Blink.xcodeproj/project.pbxproj b/Blink.xcodeproj/project.pbxproj
index 4eaa4497..9d513cfb 100644
--- a/Blink.xcodeproj/project.pbxproj
+++ b/Blink.xcodeproj/project.pbxproj
@@ -157,7 +157,6 @@
 		BD9EA217271F846100874007 /* BlinkLogging.swift in Sources */ = {isa = PBXBuildFile; fileRef = BD9EA20A271F62ED00874007 /* BlinkLogging.swift */; };
 		BD9EA218271F846400874007 /* Publisher.swift in Sources */ = {isa = PBXBuildFile; fileRef = BD9EA20C271F664D00874007 /* Publisher.swift */; };
 		BDACC7752A6F100D00D0B261 /* TrialNotification.swift in Sources */ = {isa = PBXBuildFile; fileRef = BDACC7742A6F100D00D0B261 /* TrialNotification.swift */; };
-		BDB72CB227A9C08500DCC446 /* StoreKit.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = BDB72CB127A9C08500DCC446 /* StoreKit.framework */; };
 		BDBFA30D2728914F00C77798 /* BlinkCode.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = BDBFA3052728914F00C77798 /* BlinkCode.framework */; };
 		BDBFA3122728914F00C77798 /* BlinkCodeTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = BDBFA3112728914F00C77798 /* BlinkCodeTests.swift */; };
 		BDBFA3132728914F00C77798 /* BlinkCode.h in Headers */ = {isa = PBXBuildFile; fileRef = BDBFA3072728914F00C77798 /* BlinkCode.h */; settings = {ATTRIBUTES = (Public, ); }; };
@@ -880,7 +879,6 @@
 		BD9EA20C271F664D00874007 /* Publisher.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Publisher.swift; sourceTree = "<group>"; };
 		BD9EA215271F83B400874007 /* BlinkLoggingTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = BlinkLoggingTests.swift; sourceTree = "<group>"; };
 		BDACC7742A6F100D00D0B261 /* TrialNotification.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TrialNotification.swift; sourceTree = "<group>"; };
-		BDB72CB127A9C08500DCC446 /* StoreKit.framework */ = {isa = PBXFileReference; lastKnownFileType = wrapper.framework; name = StoreKit.framework; path = System/Library/Frameworks/StoreKit.framework; sourceTree = SDKROOT; };
 		BDB8BEA726E008190093BF48 /* OwnAlertController.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = OwnAlertController.swift; sourceTree = "<group>"; };
 		BDBFA3052728914F00C77798 /* BlinkCode.framework */ = {isa = PBXFileReference; explicitFileType = wrapper.framework; includeInIndex = 0; path = BlinkCode.framework; sourceTree = BUILT_PRODUCTS_DIR; };
 		BDBFA3072728914F00C77798 /* BlinkCode.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = BlinkCode.h; sourceTree = "<group>"; };
@@ -1347,7 +1345,6 @@
 				D22277E12A26115300D4C708 /* BlinkSnippets.framework in Frameworks */,
 				B700AED81DD0F2C200100EBF /* CloudKit.framework in Frameworks */,
 				D2334F0425C1C28F00385378 /* tar.xcframework in Frameworks */,
-				BDB72CB227A9C08500DCC446 /* StoreKit.framework in Frameworks */,
 				D2DC5D4B295DB43D007E2B9D /* RevenueCat in Frameworks */,
 				D2A9B2F7272E6F26009FCBDE /* BlinkCode.framework in Frameworks */,
 				D2334EF225C1C28F00385378 /* files.xcframework in Frameworks */,
@@ -1447,7 +1444,6 @@
 			isa = PBXGroup;
 			children = (
 				D2771510287F0EA200D31F4E /* libbuild_cli.a */,
-				BDB72CB127A9C08500DCC446 /* StoreKit.framework */,
 				D248EC542689EAD7009FD817 /* SSHConfig */,
 				D2334ECF25C1C04700385378 /* awk.xcframework */,
 				D2334F0A25C1C3A000385378 /* OpenSSH.xcframework */,
diff --git a/Blink/Blink.entitlements b/Blink/Blink.entitlements
index d4f2d4ac..0cf05921 100644
--- a/Blink/Blink.entitlements
+++ b/Blink/Blink.entitlements
@@ -2,35 +2,8 @@
 <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
 <plist version="1.0">
 <dict>
-	<key>aps-environment</key>
-	<string>development</string>
-	<key>com.apple.developer.associated-domains</key>
-	<array>
-		<string>webcredentials:blink.sh</string>
-	</array>
 	<key>com.apple.developer.default-data-protection</key>
 	<string>NSFileProtectionComplete</string>
-	<key>com.apple.developer.icloud-container-identifiers</key>
-	<array>
-		<string>iCloud.$(CLOUD_ID)</string>
-	</array>
-	<key>com.apple.developer.icloud-services</key>
-	<array>
-		<string>CloudKit</string>
-		<string>CloudDocuments</string>
-	</array>
-	<key>com.apple.developer.ubiquity-container-identifiers</key>
-	<array>
-		<string>iCloud.$(CLOUD_ID)</string>
-	</array>
-	<key>com.apple.developer.ubiquity-kvstore-identifier</key>
-	<string>$(TeamIdentifierPrefix)$(CFBundleIdentifier)</string>
-	<key>com.apple.developer.user-fonts</key>
-	<array>
-		<string>app-usage</string>
-	</array>
-	<key>com.apple.developer.web-browser</key>
-	<true/>
 	<key>com.apple.security.app-sandbox</key>
 	<true/>
 	<key>com.apple.security.application-groups</key>

[l2dy]

I tested it on the TestFlight version, and could not reproduce this issue.

Could it be the entitlements I removed that caused the issue? I don't have a paid developer account, so I had to remove these:

Tested again on iPad emulator and reproduced the issue without my modifications, so it's likely not relevant.

[carloscabanero]

When I tested this myself a while ago, there was only one device where the bar was not showing up. But the TestFlight version was still working there. I will leave this open and will try to take a deeper look later if someone else has the same issue.

[l2dy]

I figured out why _keyboardWillChangeFrame was disabled. It's because _keyboardWillShow and _keyboardWillHide covers all relavent events, and they have code that is duplicate of _keyboardWillChangeFrame, with one exception that KBTracker.shared.detectHardwareKBWithSoftwareKBHeight(height: kbEndFrame.height) is missing.

I have updated the PR accordingly, and added comments about this.