Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade dependencies on v2 banch #1992

Open
wants to merge 4 commits into
base: v2
Choose a base branch
from

Conversation

mfmarche
Copy link
Contributor

@mfmarche mfmarche commented Nov 9, 2024

Fixes #1969

The main goal was upgrading werkzeug for CVE-2024-34069.
After switching to python 3.12, it proved more difficult with changes to
setuptools, etc. I decided to pull the pyproject from the main, and
utilize that, alone with updated dependencies. Small changes were needed
in various api changes, notably:

- flask change of request_ctx
- swagger_ui_bundle version change, default_template_dir change
- aiohttp middleware api slightly changed
- flask json change, using flask.json.provider

I believe these changes will have minimal impact to users, but the
changes are likely breaking for some, specifically, the move to latest
flask.

The main goal was upgrading werkzeug for CVE-2024-34069.
After switching to python 3.12, it proved more difficult with changes to
setuptools, etc. I decided to pull the pyproject from the main, and
utilize that, alone with updated dependencies. Small changes were needed
in various api changes, notably:

- flask change of request_ctx
- swagger_ui_bundle version change, default_template_dir change
- aiohttp middleware api slightly changed
- flask json change, using flask.json.provider

I believe these changes will have minimal impact to users, but the
changes are likely breaking for some, specifically, the move to latest
flask.

fixes spec-first#1969

Signed-off-by: Mike Marchetti <[email protected]>
aiohttp has conflicting requirements for latest updates, where python3.8
is deprecated. Remove 3.8 support to simplify the requirements to take
latest.

Signed-off-by: Mike Marchetti <[email protected]>
@mfmarche
Copy link
Contributor Author

mfmarche commented Nov 9, 2024

I tried to fixup the readthedocs build, i'm not sure how to resolve the issue. It could be a conflicting dependency? Is this build job required on this branch?

Copy link
Member

@RobbeSneyders RobbeSneyders left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @mfmarche!

I haven't gone through the code changes in detail yet, but the passing tests with limited changes look promising!

The docs pipeline is configured in this repo in .readthedocs.yaml, the dependencies are defined in pyproject.toml. The pipeline runs on ReadTheDocs. I suspect the issue is due to a mismatch in sphinx and sphinx-autoapi versions (logs).

I don't want to break people's projects relying on Connexion 2 by releasing breaking changes. However I am open to releasing this fix under the connexion2 pypi name instead.

README.md Outdated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should keep the old README on this branch, so the information and examples match the code.

Dockerfile3.9 Outdated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this file used?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume we can remove setup.cfg and setup.py now?

Comment on lines +17 to +20
3.9: py39-pypi
3.10: py310-pypi
3.11: py311-pypi,pre-commit
3.12: py312-pypi
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The github actions workflow should be updated to match this.

pyproject.toml Outdated
PyYAML = ">= 5.1"
requests = ">= 2.27"
typing-extensions = ">= 4.6.1"
werkzeug = ">= 2.2.1"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's bump this to a version with the CVE fixes, since this was the main reason for this PR.
3.1.3 is the version used in the 3.9 workflow with passing tests.

- fixup doc genenerator, using config from main. Too many issues
  encountered trying to get a working autoapi version to work with
sphinx. Decided to adopt a more recent version of sphinx and utilized
Connexion/main's docs/conf.py file.
- add pipeline builds for supported python
- snap werkzeug version to address CVE's

Signed-off-by: Mike Marchetti <[email protected]>
@mfmarche
Copy link
Contributor Author

thanks @RobbeSneyders for the review. I addressed all issues I believe.

@ykharko
Copy link

ykharko commented Dec 1, 2024

Hey guys, sorry for chasing. What's left here? Is there a chance to get it merged and released? I believe many people are waiting for werkzeug updates because of CVEs.

Are thare any blockers here and is there a chance to release it in the nearest future?

@mfmarche
Copy link
Contributor Author

mfmarche commented Dec 2, 2024

Hi @ykharko, I believe all issues were resolved, just waiting for final review and hopefully it can be released. thanks.

@noirbee
Copy link
Contributor

noirbee commented Dec 31, 2024

Hi @mfmarche , thanks a lot for this, started working on this for the same reasons but your work is much further along, especially in terms of CI / packaging etc.

I've started testing our code base against your branch, and the first breakage I've come across is the fact that the FlaskJSONEncoder now inherits from Python's own default JSONEncoder; i.e. it does not end up calling flask.json.provider._default(o) after falling through the few types connexion explicitly handles.

Arguably the encoder could be removed as I'm not sure it was intended to be part of connexion's actual module API, unfortunately it's used by other projects, notably openapi-code-generator's flask template, which we also use.

Would you consider the following patch against your branch to keep backwards compatibility ?

diff --git a/connexion/apps/flask_app.py b/connexion/apps/flask_app.py
index e9efaef..9814a89 100644
--- a/connexion/apps/flask_app.py
+++ b/connexion/apps/flask_app.py
@@ -153,10 +153,8 @@ class FlaskApp(AbstractApp):


 class FlaskJSONProvider(DefaultJSONProvider):
-    def __init__(self, app):
-        super().__init__(app)
-
-    def default(self, o):
+    @classmethod
+    def default(cls, o):
         if isinstance(o, datetime.datetime):
             if o.tzinfo:
                 # eg: '2015-09-25T23:14:42.588601+00:00'
@@ -177,22 +175,7 @@ class FlaskJSONProvider(DefaultJSONProvider):

 class FlaskJSONEncoder(json.JSONEncoder):
     def default(self, o):
-        if isinstance(o, datetime.datetime):
-            if o.tzinfo:
-                # eg: '2015-09-25T23:14:42.588601+00:00'
-                return o.isoformat('T')
-            else:
-                # No timezone present - assume UTC.
-                # eg: '2015-09-25T23:14:42.588601Z'
-                return o.isoformat('T') + 'Z'
-
-        if isinstance(o, datetime.date):
-            return o.isoformat()
-
-        if isinstance(o, Decimal):
-            return float(o)
-
-        return json.JSONEncoder.default(self, o)
+        return FlaskJSONProvider.default(o)


 class NumberConverter(werkzeug.routing.BaseConverter):

(I can also open a separate PR against your branch if you'd rather review it separately)

Update from @noirbee to ensure that flask.json.provider.DefaultJSONProvider.default is called.
@mfmarche
Copy link
Contributor Author

mfmarche commented Jan 2, 2025

thanks @noirbee for providing the feedback. I have updated the MR with your suggested changes.

@mfmarche
Copy link
Contributor Author

mfmarche commented Jan 2, 2025

If I'm missing something else @RobbeSneyders please let me know. I wasn't sure how you would like to publish to connexion2, if you want to change the pyproject.toml? Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants