[SDP-1397] Circle payouts pt1 #486

marcelosalloum · 2024-12-05T00:43:33Z

What

Replace the usage of Circle's POST /transfers API endpoint with the combination of POST /recipients and POST /payouts API endpoint.

Why

Circle plans to stop support for the /transfers API endpoint.

Pending

Errors in the POST /recipient endpoint are not being properly handled. What;s missing?

When POST /recipient fails with ErrCircleRecipientCreationFailedTooManyTimes, it should update the payments table with the failed status
When a payment is retried, recipients should be retried as well, either by creating a new circle_recipient in our DB, or by cleaning the sync_attempts counter from the existing recipient (TBD).
Split logic from ensureRecipientIsReady as suggested in [SDP-1397] Circle payouts pt1 #486 (comment).

Checklist

PR Structure

This PR has a reasonably narrow scope (if not, break it down into smaller PRs).
This PR title and description are clear enough for anyone to review it.
This PR does not mix refactoring changes with feature changes (split into two PRs otherwise).

Thoroughness

This PR adds tests for the new functionality or fixes.
This PR contains the link to the Jira ticket it addresses.

Configs and Secrets

No new CONFIG variables are required -OR- the new required ones were added to the helmchart's values.yaml file.
No new CONFIG variables are required -OR- the new required ones were added to the deployments (pr-preview, dev, demo, prd).
No new SECRETS variables are required -OR- the new required ones were mentioned in the helmchart's values.yaml file.
No new SECRETS variables are required -OR- the new required ones were added to the deployments (pr-preview secrets, dev secrets, demo secrets, prd secrets).

Release

This is not a breaking change.
This is ready for production.. If your PR is not ready for production, please consider opening additional complementary PRs using this one as the base. Only merge this into develop or main after it's ready for production!

Deployment

Does the deployment work after merging?

stellar-jenkins · 2024-12-05T00:46:39Z

stellar-disbursement-platform-backend-preview is available here:
SDP: https://sdp-backend-pr486.previews.kube001.services.stellar-ops.com/health
AP: https://sdp-ap-pr486.previews.kube001.services.stellar-ops.com/health
Frontend: https://sdp-backend-dashboard-pr486.previews.kube001.services.stellar-ops.com

stellar-jenkins · 2024-12-05T23:30:41Z

stellar-disbursement-platform-backend-preview is available here:
SDP: https://sdp-backend-pr486.previews.kube001.services.stellar-ops.com/health
AP: https://sdp-ap-pr486.previews.kube001.services.stellar-ops.com/health
Frontend: https://sdp-backend-dashboard-pr486.previews.kube001.services.stellar-ops.com

stellar-jenkins · 2024-12-06T01:24:43Z

stellar-disbursement-platform-backend-preview is available here:
SDP: https://sdp-backend-pr486.previews.kube001.services.stellar-ops.com/health
AP: https://sdp-ap-pr486.previews.kube001.services.stellar-ops.com/health
Frontend: https://sdp-backend-dashboard-pr486.previews.kube001.services.stellar-ops.com

stellar-jenkins · 2024-12-06T01:37:32Z

stellar-disbursement-platform-backend-preview is available here:
SDP: https://sdp-backend-pr486.previews.kube001.services.stellar-ops.com/health
AP: https://sdp-ap-pr486.previews.kube001.services.stellar-ops.com/health
Frontend: https://sdp-backend-dashboard-pr486.previews.kube001.services.stellar-ops.com

stellar-jenkins · 2024-12-06T18:27:10Z

stellar-disbursement-platform-backend-preview is available here:
SDP: https://sdp-backend-pr486.previews.kube001.services.stellar-ops.com/health
AP: https://sdp-ap-pr486.previews.kube001.services.stellar-ops.com/health
Frontend: https://sdp-backend-dashboard-pr486.previews.kube001.services.stellar-ops.com

stellar-jenkins · 2024-12-06T23:14:37Z

stellar-disbursement-platform-backend-preview is available here:
SDP: https://sdp-backend-pr486.previews.kube001.services.stellar-ops.com/health
AP: https://sdp-ap-pr486.previews.kube001.services.stellar-ops.com/health
Frontend: https://sdp-backend-dashboard-pr486.previews.kube001.services.stellar-ops.com

stellar-jenkins · 2024-12-07T01:07:42Z

stellar-disbursement-platform-backend-preview is available here:
SDP: https://sdp-backend-pr486.previews.kube001.services.stellar-ops.com/health
AP: https://sdp-ap-pr486.previews.kube001.services.stellar-ops.com/health
Frontend: https://sdp-backend-dashboard-pr486.previews.kube001.services.stellar-ops.com

stellar-jenkins · 2024-12-07T01:08:01Z

Something went wrong with PR preview build please check

…rt reconciliation with both transfer or payouts, whichever is available.

…able to reconcile both transfers and/or payouts.

…uccess

…ody.

stellar-jenkins · 2024-12-10T23:42:25Z

Something went wrong with PR preview build please check

stellar-jenkins · 2024-12-10T23:45:49Z

stellar-disbursement-platform-backend-preview is available here:
SDP: https://sdp-backend-pr486.previews.kube001.services.stellar-ops.com/health
AP: https://sdp-ap-pr486.previews.kube001.services.stellar-ops.com/health
Frontend: https://sdp-backend-dashboard-pr486.previews.kube001.services.stellar-ops.com

### What Errors in the POST /recipient endpoint are not being properly handled. What;s missing? - When POST /recipient fails with `ErrCircleRecipientCreationFailedTooManyTimes`, we update the payments table with the failed status explaining the reason. - When a payment that failed due to recipient creation issues is retried, the recipient is reset with `status=NULL` and `sync_attempts=0`. - The method `ensureRecipientIsReady` was split into different sub-methods for better modularity and clarity, as suggested in #486 (comment). <img width="1151" alt="Screenshot 2024-12-12 at 4 39 08 PM" src="https://github.com/user-attachments/assets/53ed59d9-bba0-42e2-b149-0fca3bfb5a66" /> - [x] Tested with scheduled jobs - [x] Tested with Kafka ### Why To wrap up https://stellarorg.atlassian.net/browse/SDP-1397

marcelosalloum self-assigned this Dec 5, 2024

marcelosalloum temporarily deployed to Anchor Integration Tests December 5, 2024 00:43 — with GitHub Actions Inactive

marcelosalloum marked this pull request as draft December 5, 2024 00:43

marcelosalloum temporarily deployed to Receiver Registration - E2E Integration Tests (Stellar) December 5, 2024 00:43 — with GitHub Actions Inactive

marcelosalloum temporarily deployed to Anchor Integration Tests December 5, 2024 23:27 — with GitHub Actions Inactive

marcelosalloum temporarily deployed to Receiver Registration - E2E Integration Tests (Stellar) December 5, 2024 23:27 — with GitHub Actions Inactive

marcelosalloum temporarily deployed to Anchor Integration Tests December 6, 2024 01:22 — with GitHub Actions Inactive

marcelosalloum temporarily deployed to Receiver Registration - E2E Integration Tests (Stellar) December 6, 2024 01:22 — with GitHub Actions Inactive

marcelosalloum force-pushed the feat/SDP-1397-circle-payouts branch from a46248e to 5652b7e Compare December 6, 2024 01:34

marcelosalloum temporarily deployed to Receiver Registration - E2E Integration Tests (Stellar) December 6, 2024 01:34 — with GitHub Actions Inactive

marcelosalloum temporarily deployed to Anchor Integration Tests December 6, 2024 01:34 — with GitHub Actions Inactive

marcelosalloum temporarily deployed to Anchor Integration Tests December 6, 2024 18:24 — with GitHub Actions Inactive

marcelosalloum temporarily deployed to Receiver Registration - E2E Integration Tests (Stellar) December 6, 2024 18:24 — with GitHub Actions Inactive

marcelosalloum temporarily deployed to Receiver Registration - E2E Integration Tests (Stellar) December 6, 2024 23:11 — with GitHub Actions Inactive

marcelosalloum temporarily deployed to Anchor Integration Tests December 6, 2024 23:11 — with GitHub Actions Inactive

marcelosalloum temporarily deployed to Anchor Integration Tests December 7, 2024 01:04 — with GitHub Actions Inactive

marcelosalloum temporarily deployed to Receiver Registration - E2E Integration Tests (Stellar) December 7, 2024 01:04 — with GitHub Actions Inactive

marcelosalloum temporarily deployed to Anchor Integration Tests December 7, 2024 01:04 — with GitHub Actions Inactive

marcelosalloum temporarily deployed to Receiver Registration - E2E Integration Tests (Stellar) December 7, 2024 01:04 — with GitHub Actions Inactive

marcelosalloum force-pushed the feat/SDP-1397-circle-payouts branch from 9f12f0d to eb34fe4 Compare December 9, 2024 22:07

marcelosalloum temporarily deployed to Receiver Registration - E2E Integration Tests (Stellar) December 9, 2024 22:07 — with GitHub Actions Inactive

marcelosalloum temporarily deployed to Anchor Integration Tests December 9, 2024 22:07 — with GitHub Actions Inactive

marcelosalloum added 17 commits December 10, 2024 15:42

[WIP] Implement Circle payment using the /payouts API, and also suppo…

87e6ce0

…rt reconciliation with both transfer or payouts, whichever is available.

Polishes & fixes.

8ae5cbc

Refactor circle_reconciliation_service.go and update its tests to be …

fc57edd

…able to reconcile both transfers and/or payouts.

Add tests to CircleRecipientModel.

89952b1

Add unit tests: Test_CirclePaymentDispatcher_ensureRecipientIsReady_s…

158259c

…uccess

Implement Test_CirclePaymentDispatcher_ensureRecipientIsReady_failure.

0ec2c7b

Remove DB constraint.

2fa0ccc

Reuse query.

301fe88

Add receiver phone and email to the payment.ReceiverWallet.Receiver b…

35c0c44

…ody.

Add indexes to the statuses commonly filtered for.

8ff533f

Add TODOs to remove methods.

ebb148c

Update test feedback.

ce0f9a6

Update delete contact info handler and improve its tests.

daa1a31

Attempt to fix test that's failing in GH action.

f179091

Bump github.com/avast/retry-go.

03facd5

Address code review suggestions.

bbcc4b2

Avoid using pointers when not needed.

0480232

marcelosalloum force-pushed the feat/SDP-1397-circle-payouts branch from a15b635 to 0480232 Compare December 10, 2024 23:42

marcelosalloum had a problem deploying to Receiver Registration - E2E Integration Tests (Stellar) December 10, 2024 23:42 — with GitHub Actions Failure

marcelosalloum temporarily deployed to Anchor Integration Tests December 10, 2024 23:42 — with GitHub Actions Inactive

marcelosalloum had a problem deploying to Receiver Registration - E2E Integration Tests (Stellar) December 10, 2024 23:45 — with GitHub Actions Failure

marcelosalloum temporarily deployed to Receiver Registration - E2E Integration Tests (Stellar) December 10, 2024 23:53 — with GitHub Actions Inactive

marcelosalloum merged commit fa51cec into develop Dec 10, 2024
13 checks passed

marcelosalloum deleted the feat/SDP-1397-circle-payouts branch December 10, 2024 23:56

marcelosalloum added a commit that referenced this pull request Dec 12, 2024

Split logic from ensureRecipientIsReady as suggested in #486 (comment)

5755d99

marcelosalloum added a commit that referenced this pull request Dec 13, 2024

Split logic from ensureRecipientIsReady as suggested in #486 (comment)

3f5bed8

marcelosalloum mentioned this pull request Dec 13, 2024

[SDP-1397] Circle Payouts pt2: error handling & retries #491

Merged

14 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SDP-1397] Circle payouts pt1 #486

[SDP-1397] Circle payouts pt1 #486

marcelosalloum commented Dec 5, 2024 •

edited

Loading

stellar-jenkins commented Dec 5, 2024

stellar-jenkins commented Dec 5, 2024

stellar-jenkins commented Dec 6, 2024

stellar-jenkins commented Dec 6, 2024

stellar-jenkins commented Dec 6, 2024

stellar-jenkins commented Dec 6, 2024

stellar-jenkins commented Dec 7, 2024

stellar-jenkins commented Dec 7, 2024

stellar-jenkins commented Dec 10, 2024

stellar-jenkins commented Dec 10, 2024

[SDP-1397] Circle payouts pt1 #486

[SDP-1397] Circle payouts pt1 #486

Conversation

marcelosalloum commented Dec 5, 2024 • edited Loading

What

Why

Pending

Checklist

PR Structure

Thoroughness

Configs and Secrets

Release

Deployment

stellar-jenkins commented Dec 5, 2024

stellar-jenkins commented Dec 5, 2024

stellar-jenkins commented Dec 6, 2024

stellar-jenkins commented Dec 6, 2024

stellar-jenkins commented Dec 6, 2024

stellar-jenkins commented Dec 6, 2024

stellar-jenkins commented Dec 7, 2024

stellar-jenkins commented Dec 7, 2024

stellar-jenkins commented Dec 10, 2024

stellar-jenkins commented Dec 10, 2024

marcelosalloum commented Dec 5, 2024 •

edited

Loading