Development Best Practices when using WithPerceptionTracking in iOS 16 and earlier.

[scogeo]

This is a follow up discussion to a slack thread that was discussing performance issues related to a recent change that enables perception checking on iOS 17 while targeting older iOS versions.

For those who are not familiar, perception checking is a cool feature that comes with the Perception framework that provides run-time warnings in Xcode when the framework detects access to ObservableState that was not contained within a WithPerceptionTracking block in SwiftUI. It is an automated way to find bugs that would otherwise be very hard to track down when using ObservableState on iOS 16 and earlier. However, these checks do come with a cost. Over the course of the beta, performance of perception tracking has improved substantially, but there is still a noticeable cost associated with it in moderately complex apps. It should be noted this performance cost is only apparent in DEBUG builds. Perception checks are not performed in RELEASE builds.

When this change was made I immediately noticed a degradation in performance of my app when running on iOS 17 in the simulator. Thoughtfully, an option is now provided to disable perception checking in the form of a global isPerceptionCheckingEnabled which can be set to false. For example:

final class AppDelegate: NSObject, UIApplicationDelegate {
  public override init() {
    isPerceptionCheckingEnabled = false
  }  
}

The question I raised is what should the default value of isPerceptionCheckingEnabled be on iOS 17 and later. Currently, it defaults to true. I suggested perhaps it should be defaulted to false.

But the real question is what is the typical best practice for developing TCA based apps using @ObservableState and running on iOS 16 and earlier?

My current workflow while experimenting with the observation-beta branch has been to develop as normal on iOS 17 in the simulator with zero performance penalties. Then I fire up an iOS 16 simulator and run through the app (with reduced performance) and use the perception checking feature to look for perception tracking issues in the form of the purple runtime warnings in Xcode. Resolve any issues and go back to developing on iOS 17. So I have been viewing this as two separate development activities.

With this recent change, I can now in theory do this concurrently. But in practice, the performance issues get in the way of normal development so I immediately disabled perception checking by default and now only turn it on when I want to do the activity of checking for perception tracking issues.

This is of course a very manual process, and one of my next steps is to update my snapshot unit tests and run those on iOS 16 to find perception issues in a more automated manner. My understanding is that the runtime perception checks will become errors when running under test. But I have yet to try this out as I haven't fully updated my testing infrastructure with perception tracking in mind.

So the questions for the community are what is the right default for isPerceptionCheckingEnabled on iOS 17 and later, and what are your workflows going to be for developing and testing ObservableState on iOS 16 and earlier?

[stephencelis]

Our general reason behind the change was we noticed that folks were missing perception checks because they were developing in iOS 17 simulators while supporting earlier devices. To avoid these kinds of platform-specific bugs from entering a release, we think the safer default is to always have perception checks enabled, and ideally continue to improve the performance. We're very interested in ideas and PRs that can address existing debug performance.

As far as your desired workflow goes, you can achieve it today with the following:

 final class AppDelegate: NSObject, UIApplicationDelegate {
   public override init() {
+    #if available(iOS 17, *) {
       isPerceptionCheckingEnabled = false
+    }
   }  
 }

This opt-in approach would allow you to to work the way you want to work and accept that pre-iOS 17 observation bugs could leak into a release, but it's unlikely due to your workflow.

[scogeo]

This is a good suggestion. But I actually like the fact that I can now test in the same iOS 17 simulator where I have already potentially loaded test data, etc. I can also test for perception issues on a device running iOS 17 now, which previously required me to have a device running iOS 16 which I don't always have readily available. So in many ways this change was good, it just disrupted my existing (manual) workflow.

In the next few weeks I'll play around with UI testing more on iOS 16 with perception checking in mind, and see how that may change how I test going forward. The biggest challenge I have is that my unit tests are very heavily focused on reducer logic and fairly light on UI testing, due to the frequency of UI churn. So I need to build out my test suite to get better coverage before I can rely on it to reliably find perception tracking issues.

[mbrandonw]

Hi @scogeo (and cc @jshier), we just pushed some changes to the observation-beta-perception-performance branch on TCA. Can you point your project to that branch when you have a moment to let us know how the performance is?

[mbrandonw]

Hi @evjand, you can pull the latest of the perception branch for a fix.

[mbrandonw]

Hi @scogeo, @jshier, @evjand: We just PR'd the changes and the code changed a bit since we needed to clean up some hackiness in the initial version. Can you give it a spin one more time just to make sure it's good to merge?

[evjand]

Still fast and working here 👍🏻

[scogeo]

@mbrandonw All good for me as well. I tested using this commit

[mbrandonw]

Thanks for testing! We also pushed a commit to observation-beta that fixes the erroneous runtime warnings that @evjand was seeing.