Repository navigation
Restore atan_difference gradients at negative-one products - #2039
Open
DiadkoShmek wants to merge 1 commit into
Open
DiadkoShmek wants to merge 1 commit into
DiadkoShmek wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
atan_differencehas the correct value atx*y == -1, but its constant selected branch removes the smooth partial derivatives while the inactive singular quotient poisons reverse AD. Both TensorFlow and JAX are affected. This reaches an ordinary finite likelihood:The correct score is
[8/17, -15/17 + 2/pi], approximately[0.470588, -0.245733]. The density value itself is already correct; forward AD also loses the normalizer's derivative.Keep the cancellation-resistant quotient away from the pole. At opposite-sign poles,
atan(x) - atan(y)is cancellation-free and supplies the correct derivatives. Use the existing native conditional to evaluate that additional work only when a pole is present. The inactive quotient denominator is made finite as well, because JAXvmapmay lower conditionals to selects.Validation:
tf.functionin four TF2 contexts; its focused BASE comparison fails all four and the candidate passes. Reverse checks remain in all regimes; the TF1 forward limitation is guarded.Two fixed, warmed, non-pole JAX CPU value-and-gradient fixtures measure scalar: 10.8455 → 11.823 µs (+0.9775µs/+9.0%) and batch of 4,096: 152.184 → 153.5765 µs (+0.9%). These are bounded local cost controls, with compilation recorded separately; they do not establish a general speed or no-regression guarantee.
The numerical scope is finite real inputs and first AD on the tested backends. Full project/Bazel/nightly, GPU, sharding, higher AD and universal IEEE behavior were not tested. The separate Lambert-W change in #2038 is not included.
I directed this work and take responsibility for the contribution and its follow-up. AI assistance was used in producing the change.