Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release v0.6.0 #368

Closed
wants to merge 61 commits into from
Closed

Release v0.6.0 #368

wants to merge 61 commits into from

Conversation

geofmureithi
Copy link
Owner

@geofmureithi geofmureithi commented Jul 13, 2024

New version alert!

  • MAJOR FEATURE: Request now has been standardized!
  • Improved external API and exports for Redis: Now supports Deadpool and other crates based on the Redis crate.
  • Exposed storage codecs.
  • Introduced new features and minor fixes.
  • Standardized backends for storage, MQ, and cron.
  • Added new examples and improved existing ones.
  • Made configuration publicly accessible.
  • Introduced a panic-catching layer.

geofmureithi and others added 30 commits May 22, 2024 22:11
Breaking Changes:
- Dropped traits Job and Message, please use namespace
* fix: introduce namespace and codec config

* fix: missing apis
* api: for redis and sqlite

* Version: 0.6.0-alpha.1

Changelog:
- Redis storage doesnt require pool to be clone. Allows use of deadpool-redis among others.
- Namespace is picked by default for `new` methods.

* fix: docs and tests

* lint: cargo clippy and fmt

* postgres: add a listener example
* api: for redis and sqlite

* Version: 0.6.0-alpha.1

Changelog:
- Redis storage doesnt require pool to be clone. Allows use of deadpool-redis among others.
- Namespace is picked by default for `new` methods.

* fix: docs and tests

* lint: cargo clippy and fmt

* postgres: add a listener example

* bump: to v0.6.0-alpha.1
* feat: remove the `Clone` requirements for services

* test save

* fix: get buffered layer working

* update: remove clone & update api

* fix: tests and api

* lint: clippy fixes

* lint: cargo fmt
* fix: backend layers were not loaded

* fix: handle clone
* fix: mq ack

* lint: fmt
* fix: handle unwraps in storages

* fix: ensure no unwrap
* fix: better apalis deps allowing tree shaking for backends

* fix: remove backend features in the root crate
* fix: standardize backend for storage and mq

* fix: minor fixes
* fix: add missing exposed config

* fix: add getters
* Feature: Add a layer that catches panics

This allows preventing job execution from killing workers and returns an error containing the backtrace

* fix: backtrace as it may be different

* add: example for catch-panic

* fix: make  not default
* Feature: Save results for storages

Currently just the status is stored, this PR adds the ability to save the result

* fix: result from storage

* fix: kill and abort issue
geofmureithi and others added 11 commits July 25, 2024 21:07
* fix: ack api to allow backend to handle differently

* fix: related to storage tests

* fix: calculate status for postgres
* chore: fix typos (#346)

* chore: Add repository to metadata (#345)

* fix(deps): update rust crate sqlx to 0.8.0

* fix: sqlite example

---------

Co-authored-by: John Vandenberg <[email protected]>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: geofmureithi <[email protected]>
Co-authored-by: Geoffrey Mureithi <[email protected]>
* bump: to v0.6.0-rc.5

* fix: standardize codec usage

* lint: cargo fmt
* add: catch-panic example

* add: graceful shutdown example

* add: unmonitored example

* add: arguments example

* fix: minor updates

* fix: sql tests

* fix: minor updates
* fix: improve on benches

* fix: bench trigger

* fix: include tokio for sqlx

* fix: improve the benching approach

* fix: mysql api

* fix: redis api

* fix: improve bench approach, remove counter

* remove: setup

* remove: pg

* fix: pg

* fix: pg
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Geoffrey Mureithi <[email protected]>
* fix: add some missing data required for dependency injection

* lint: clippy and fmt
@jayvdb
Copy link
Contributor

jayvdb commented Sep 7, 2024

Could we sneak #413 into this release.
It pushes out a lot of really old dependencies from the dep tree. c.f. async-rs/async-std#1081 -

jayvdb and others added 7 commits September 9, 2024 07:41
* wip: introduce context to request

* fix: get request context working

* lint: cargo fmt

* fix: get tests compiling

* add: push_request and shedule_request

* fix: task_id for Testwrapper

* fix: minor checks and fixes on postgres tests

* fix: bug on postgres fetch_next
Problem: We are missing crucial `FromRequest` impls for:
- TaskId
- Attempt
- Namespace

Also removed `Context<E>`

Solution: Implement `FromRequest` for these Types.
* fix:[bug] include backend provided layer in service layers.

Problem:
The current worker logic is missing an implementation where the backend provided layer should be added to the service's layer.
This is a critical issue that affects all v0.6.0-rc-7 users and they should update as soon as a new release is done.

Solution:
- Add backend layers to service's layer.
- Add worker_consume tests on the storages to prevent regression on this.

* chore: comment an enforcement rule not yet followed by redis
So that a custom number of attempts can be configured:

  let mut ctx = SqlContext::new();
  ctx.set_max_attempts(2);
  let req = Request::new_with_ctx(job, ctx);
  storage.push_request(req).await.unwrap();

While the default is still to try up to 25 times:

  storage.push(job).await.unwrap();
@victoronascimento
Copy link

I've been following this project from afar. Jobs done this way seem like a very important tool and trustworthy tool to have in the community. This library seems like it will be what https://getoban.pro/ is for the Elixir community. Thanks a lot for all your work @geofmureithi !!

From what I can read from the code, this version is the one that will make the postgres adapter use LISTEN/NOTIFY (using a PgSubscription), right? I think this is necessary to compare it to Oban and check on performance benchmarks.

If it helps, Oban has evolved a lot through the time and many tricks there might be applicable here like:

Do you have a time frame on your mind for releasing 0.6.0?

@geofmureithi
Copy link
Owner Author

@victoronascimento Thanks for highlighting Oban, (I have been looking alot at Sidekiq and Celery) and this seems a good reference too.
I plan to do better benchmarks on https://github.com/geofmureithi/apalis-benchmarks.

Regarding the release, I need to finalize on #428 then we should all be good. You can still use the rc candidates for now.
Let me try and do the release within a week.

zakstucke and others added 7 commits November 19, 2024 15:38
Making sqlx accessible to users of apalis without requiring them to
explicitly add it as a dependency.
* feat: introducing WorkerBuilderExt which makes the work of building a new worker way easier.

* improve: worker api almost there

* fix: radical improvements and updates. Removed executor and got graceful shutdown working

* chore: deprecate register with count and force builder order

* chore: more improvements on the worker

* fix: allow DI for Worker

* add: get the task count by a worker

* lint: fmt and clippy

* fix: allow worker stopping
* fix: relax the api provided for sqlx req

* lint: clippy and fmt
* feat: add recovery of abandoned jobs to backend heartbeats

* lint: fmt

* fix: attempt to get tests passing

* fix: attempt to get tests passing

* fix: minor fix typo

* fix: minor different solutions

* fix: better handle attempts

* handle postgres edge case

* fix: better handling
* feat: allow backends to emit errors

* lint: fmt

* fix: pass in a reference to prevent mutation
* Feat: Introduce simple ability to pipe cron jobs to any backend

This feature allows you to quickly persist cron jobs guaranteeing they will be run and can be distributed

* lint: cargo fmt
@geofmureithi geofmureithi deleted the branch master November 26, 2024 06:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants