diff --git a/src/post/2025/index.md b/src/post/2025/index.md new file mode 100644 index 0000000..4b7f897 --- /dev/null +++ b/src/post/2025/index.md @@ -0,0 +1 @@ +* [Adding Python applications to a Python environment in Nixpkgs](./nixpkgs-python-applications.md) diff --git a/src/post/2025/nixpkgs-python-applications.md b/src/post/2025/nixpkgs-python-applications.md new file mode 100644 index 0000000..73b6e7c --- /dev/null +++ b/src/post/2025/nixpkgs-python-applications.md @@ -0,0 +1,42 @@ +--- +title: Adding Python applications to a Python environment in Nixpkgs +feed: post +--- + +Typically, when I want to assemble a Python environment containing a specific set of packages, I use nixpkgs and the [`python3.withPackages`](https://nixos.org/manual/nixpkgs/stable/#python.withpackages-function) function: + + pkgs.python3.withPackages (p: with p; [ beautifulsoup4 flask ]) + +where `withPackages` takes a function over the set of Python packages for the Python version. For example, in release 24.11, `python3` is an alias for `python312`, so `p` in the above is `python312Packages`. The above produces a `result` symlink with the expected Python environment, including both the libraries and their wrapper scripts: + + $ nix build --impure --expr 'with import {}; pkgs.python3.withPackages (p: with p; [ beautifulsoup4 flask ])' + $ ls result/bin/ + 2to3 2to3-3.12 chardetect flask idle idle3 idle3.12 normalizer pydoc pydoc3 pydoc3.12 python python3 python3.12 python3.12-config python3-config python-config + $ ./result/bin/python3 -c "import bs4; print(bs4)" + + $ ./result/bin/python3 -c "import flask; print(flask)" + + +Note that `flask` is available as a library and also through its wrapper script in `result/bin`. + +However, several Python libraries are considered to be end user applications, and so nixpkgs packages them as top-level packages and not as `pythonPackages` members. For example, the `mloader` package, a manga scraper CLI written in Python, is available as `pkgs.mloader` rather than `pkgs.python3Packages.mloader`. Consequently, the following does not work: + + $ nix build --impure --expr 'with import {}; pkgs.python3.withPackages (p: with p; [ mloader ])' + $ ls result/bin + 2to3 2to3-3.12 idle idle3 idle3.12 pydoc pydoc3 pydoc3.12 python python3 python3.12 python3.12-config python3-config python-config + $ ./result/bin/python3 -c "import mloader; print(mloader)" + Traceback (most recent call last): + File "", line 1, in + ModuleNotFoundError: No module named 'mloader' + +As I understand, this is because `pythonPackages` members are created with `pkgs.buildPythonPackage`, which is made to enable composition into environments, whereas `mloader` and other end-user programs are built with `buildPythonApplication`, which is not. The two builders are defined in the same place in nixpkgs, so learning why is tractable, but not the subject of this post. + +Like all good things in nixpkgs, this behavior can be overridden with `pkgs.python3Packages.toPythonModule`, which converts the application back into a composable Python module: + + $ nix build --impure --expr 'with import {}; pkgs.python3.withPackages (p: [(pkgs.python3Packages.toPythonModule pkgs.mloader)])' + $ ls result/bin + 2to3 2to3-3.12 idle idle3 idle3.12 mloader normalizer pydoc pydoc3 pydoc3.12 python python3 python3.12 python3.12-config python3-config python-config + $ ./result/bin/python3 -c "import mloader; print(mloader)" + + +Now our application is installed in the environment and importable, the same way it would be if we had used `python3 -m venv` to install the package directly. diff --git a/src/post/index.md b/src/post/index.md index fa8a9c7..93ef28b 100644 --- a/src/post/index.md +++ b/src/post/index.md @@ -4,6 +4,7 @@ title: Posts [RSS](./feed.xml) +* [Adding Python applications to a Python environment in Nixpkgs](./2025/nixpkgs-python-applications.md) * [Lesser-known operators in C#](./2024/lesser-known-operators-in-cs.md) * [Archiving bookmarks](./2024/bookmarks.md) * [SHLVL PS1](./2023/shlvl.md)