Skip to content
JustTryingToGetSomeWorkDonePublic

About

Deterministic Python runtime and package selection without virtual environments.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

74 Commits

Folders and files

Repository files navigation

NodePhell

NodePhell

No dependency hell: shared Python runtimes and packages without virtual environments.

NodePhell is a shared Python runtime and package system for Linux.

Projects declare the Python version and packages they need. NodePhell obtains the missing pieces, verifies them, preserves exact runtimes and package artifacts for reuse, and selects the correct combination automatically whenever the project runs.

After setup, normal use is simply:

python app.py

There is no virtual environment to activate, no private dependency tree to rebuild for every project, and no need to modify the operating system's Python.

Old projects can keep using the Python generation they were written for. New projects can use newer releases. Compatible projects reuse the same stored runtimes and package artifacts instead of installing unnecessary copies.

NodePhell can also extend the same model to Linux applications that embed Python. Adapter plugins let those applications use locked package sets, reuse exact packages they already own, and obtain missing packages without putting application-specific behavior into NodePhell core.

Version 0.1.0 is the first public release. Its feature set and release validation are complete; later work is driven by real use rather than an open prerelease checklist.

Why NodePhell is different

Python dependency conflicts are usually handled by building a separate environment around each project:

choose Python
- create an environment
- activate it
- install another dependency set
- keep the correct environment selected
- rebuild or repair it when necessary

NodePhell moves that work into the computer:

project metadata
- exact Python runtime
- exact package artifacts
- shared verified storage
- automatic selection at launch

The project still gets an exact, deterministic package set. The difference is that isolation comes from selection, not from giving every project its own private copy of a Python environment.

NodePhell is not another environment format. It is a shared store plus a selection layer.

One Python history, shared

NodePhell treats interpreters and package releases as reusable artifacts.

A project that needs an older Python release does not have to move just because another project uses a newer one. NodePhell can preserve multiple CPython runtimes and select the exact runtime recorded by each project's lock.

Packages work the same way. If an exact compatible artifact already exists, NodePhell reuses it instead of downloading another copy. Pure-Python packages may be shared across compatible Python versions, while incompatible native builds remain separate automatically.

                         shared NodePhell store

                    CPython 3.11 ---------- project A
                                        +-- project B

                    CPython 3.13 ---------- project C

                    lark 1.3.0 ------------ project A
                                        +-- project B
                                        +-- project C

                    requests 2.32 --------- project A
                                        +-- project C

                    NumPy cp311 build ----- project A
                                        +-- project B

                    NumPy cp313 build ----- project C

verified external
application package ----------------------- application D

Compatible projects reuse the same stored interpreter and package artifacts. NodePhell only keeps separate copies when compatibility actually requires them, such as different Python ABIs or platform-specific native builds.

This lets NodePhell preserve more of Python's history while often storing fewer duplicate files than per-project environments.

Everyday use

NodePhell itself currently requires Python 3.11 or newer.

Install the downloaded release wheel with pipx, optionally add the FreeCAD adapter wheel, then install NodePhell's launchers once:

pipx install ./nodephell-0.1.0-py3-none-any.whl
pipx inject nodephell ./nodephell_freecad_adapter-0.1.0-py3-none-any.whl
nodephell launcher install
nodephell --version

The adapter is optional. From a source checkout, the equivalent development installation is:

./bin/nodephell launcher install
nodephell --version

When another Python command has priority, the installer can put ~/.local/bin first in Bash startup. Close the terminal and open a new one after accepting that change; the installer also prints commands for people who prefer to refresh the current terminal.

For a new project:

nodephell init

For an existing project with pyproject.toml:

nodephell sync

If the project declares a standard Python [build-system], synchronization also connects the live source checkout to the selected Python. Code changes are used directly, while the project can report its installed name and version and provide its own commands normally. The small support files live in NodePhell's central ~/.python/source-projects area rather than a .venv inside the project or either immutable store.

Projects may also declare standard optional features and dependency groups. Select them without invoking pip directly:

nodephell options

The terminal checklist records the choices in the generated project lock and synchronizes the resulting dependency set. Deselecting a feature changes only this project. Shared releases are kept by default, are never removed while another project uses them, and require separate confirmation before safe removal.

Projects using Poetry metadata are also supported. NodePhell translates older [tool.poetry.dependencies] runtime declarations and exposes both [tool.poetry.group.NAME.dependencies] and the older [tool.poetry.dev-dependencies] form through nodephell options. Standard and Poetry groups may be used together.

PEP 621 projects whose build backend supplies dynamic = ["dependencies"] are supported as well. Their Python requirement and optional features must remain static so NodePhell can select a runtime and present options before invoking the backend.

If synchronization succeeds but a project command fails, run the guided diagnostic:

nodephell troubleshoot

NodePhell suggests declared project commands when possible, reproduces the failure, and then tries the earliest Python minor line allowed by the project's range. It preserves the original lock and asks before keeping a successful trial. For a repeatable or non-interactive command, place it after --, as in nodephell troubleshoot -- frogmouth --help.

After that, use Python normally:

python app.py
python3 -m unittest
pytest

There is nothing to activate.

NodePhell uses standard project metadata. A typical project might contain:

[project]
name = "example"
version = "0.1.0"
requires-python = ">=3.13,<3.14"
dependencies = [
    "lark",
    "requests>=2.32,<3",
]

pyproject.toml expresses human intent. NodePhell writes standard pylock.toml when the package lock conforms to PEP 751. When reproducibility also requires NodePhell-specific interpreter or embedded-host artifacts, it writes nodephell.lock.toml instead. Both forms record exact package artifacts and hashes; NodePhell reads either filename and refuses an ambiguous project containing both.

When requirements change, run:

nodephell sync

What happens when Python starts

Inside a NodePhell project, the launcher:

  1. finds the project's lock;
  2. selects the exact locked CPython runtime;
  3. finds each locked package from an approved provider;
  4. builds the required package view;
  5. starts the process.

A package may come from:

  • NodePhell's managed package store;
  • the selected Python runtime; or
  • a verified external application package store.

If an exact compatible copy already exists and can be verified, NodePhell can reuse it instead of downloading another one.

Project launches exclude the generic Python user site and inherited PYTHONPATH, preventing unrelated local packages from silently contaminating the locked package set.

Outside a NodePhell project, python and python3 delegate to the operating system's Python unchanged.

Package commands without activation

Python distributions often provide commands through console_scripts.

NodePhell preserves that behavior without a virtual environment. If the selected package set provides pytest, for example:

pytest

works normally.

NodePhell-managed shims resolve the calling project at invocation time, so different projects can use different versions of the same command without activating different shells or environments.

Only commands declared by valid Python distribution metadata are exposed. Arbitrary executables found in an application directory are not turned into NodePhell commands.

Python applications can use NodePhell too

NodePhell is not limited to programs started with python.

Many Linux applications embed Python and maintain their own package directories. An adapter plugin lets such an application participate in the same runtime and package-selection system.

An adapter describes only application-specific facts, such as:

  • how to identify the application's embedded Python;
  • which Python ABI and platform it uses;
  • which package directories it already owns;
  • how additional package paths are supplied; and
  • how command-line or graphical modes are launched.

NodePhell core continues to handle package selection, ABI matching, hashes, shared storage, project references, and execution.

This also lets NodePhell reuse exact verified packages that an application already has instead of downloading duplicate copies. Missing locked packages can be supplied separately without modifying the application's own package store.

FreeCAD is the current reference adapter. It is a plugin, not a special case in NodePhell core.

Adapters may be installed as Python entry points or discovered from NodePhell's standard local plugin location. See the adapter documentation for authoring, packaging, scanning, and validation details.

Status

Version 0.1.0 is released with the initial feature set complete. Future work will respond to demonstrated compatibility needs and user feedback.

Current validation includes ordinary and editable source projects, historical runtimes, native packages and ABI boundaries, package commands, shared package reuse, cleanup and integrity checking, external package stores, and embedded Python applications.

Current platform scope

  • NodePhell itself requires Python 3.11 or newer.
  • Automatic CPython and FreeCAD artifact acquisition currently targets supported Linux builds.
  • Direct dependencies may omit a version, specify an exact version, or use a version range.
  • Build backends may supply dynamic PEP 621 dependencies; sync refreshes them each time.
  • Standard PEP 508 dependency markers are evaluated by the selected Python runtime.
  • Generated locks always record exact package releases.
  • Direct URL or path requirements are not yet supported.
  • FreeCAD is the current reference embedded-host adapter.

Documentation

Run nodephell --help or add --help after a command group for the complete command reference.

License

NodePhell is licensed under the GNU General Public License, version 3 only.

SPDX-License-Identifier: GPL-3.0-only

About

Deterministic Python runtime and package selection without virtual environments.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages