Jaculabilis/www
1
0
Fork 0
Code

toPythonModule post

Browse Source
This commit is contained in:
Tim Van Baak 2025-03-05 07:27:50 -08:00
parent 76c6c5ae01
commit 4b64376e7d
3 changed files with 44 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
src/post/2025/index.md Normal file
View File

@ -0,0 +1 @@
* [Adding Python applications to a Python environment in Nixpkgs](./nixpkgs-python-applications.md)

42
src/post/2025/nixpkgs-python-applications.md Normal file
View File

@ -0,0 +1,42 @@
---
title: Adding Python applications to a Python environment in Nixpkgs
feed: post
---
Typically, when I want to assemble a Python environment containing a specific set of packages, I use nixpkgs and the [`python3.withPackages`](https://nixos.org/manual/nixpkgs/stable/#python.withpackages-function) function:
pkgs.python3.withPackages (p: with p; [ beautifulsoup4 flask ])
where `withPackages` takes a function over the set of Python packages for the Python version. For example, in release 24.11, `python3` is an alias for `python312`, so `p` in the above is `python312Packages`. The above produces a `result` symlink with the expected Python environment, including both the libraries and their wrapper scripts:
$ nix build --impure --expr 'with import <nixpkgs> {}; pkgs.python3.withPackages (p: with p; [ beautifulsoup4 flask ])'
$ ls result/bin/
2to3 2to3-3.12 chardetect flask idle idle3 idle3.12 normalizer pydoc pydoc3 pydoc3.12 python python3 python3.12 python3.12-config python3-config python-config
$ ./result/bin/python3 -c "import bs4; print(bs4)"
<module 'bs4' from '/nix/store/pdbs94f0mkah2bmdz7dg6k0xcpkj7h53-python3-3.12.7-env/lib/python3.12/site-packages/bs4/__init__.py'>
$ ./result/bin/python3 -c "import flask; print(flask)"
<module 'flask' from '/nix/store/pdbs94f0mkah2bmdz7dg6k0xcpkj7h53-python3-3.12.7-env/lib/python3.12/site-packages/flask/__init__.py'>
Note that `flask` is available as a library and also through its wrapper script in `result/bin`.
However, several Python libraries are considered to be end user applications, and so nixpkgs packages them as top-level packages and not as `pythonPackages` members. For example, the `mloader` package, a manga scraper CLI written in Python, is available as `pkgs.mloader` rather than `pkgs.python3Packages.mloader`. Consequently, the following does not work:
$ nix build --impure --expr 'with import <nixpkgs> {}; pkgs.python3.withPackages (p: with p; [ mloader ])'
$ ls result/bin
2to3 2to3-3.12 idle idle3 idle3.12 pydoc pydoc3 pydoc3.12 python python3 python3.12 python3.12-config python3-config python-config
$ ./result/bin/python3 -c "import mloader; print(mloader)"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'mloader'
As I understand, this is because `pythonPackages` members are created with `pkgs.buildPythonPackage`, which is made to enable composition into environments, whereas `mloader` and other end-user programs are built with `buildPythonApplication`, which is not. The two builders are defined in the same place in nixpkgs, so learning why is tractable, but not the subject of this post.
Like all good things in nixpkgs, this behavior can be overridden with `pkgs.python3Packages.toPythonModule`, which converts the application back into a composable Python module:
$ nix build --impure --expr 'with import <nixpkgs> {}; pkgs.python3.withPackages (p: [(pkgs.python3Packages.toPythonModule pkgs.mloader)])'
$ ls result/bin
2to3 2to3-3.12 idle idle3 idle3.12 mloader normalizer pydoc pydoc3 pydoc3.12 python python3 python3.12 python3.12-config python3-config python-config
$ ./result/bin/python3 -c "import mloader; print(mloader)"
<module 'mloader' from '/nix/store/myxczrxjpsznspikp5w768c6a4zfl2iz-python3-3.12.7-env/lib/python3.12/site-packages/mloader/__init__.py'>
Now our application is installed in the environment and importable, the same way it would be if we had used `python3 -m venv` to install the package directly.

1
src/post/index.md
View File

@ -4,6 +4,7 @@ title: Posts
[RSS](./feed.xml)
* [Adding Python applications to a Python environment in Nixpkgs](./2025/nixpkgs-python-applications.md)
* [Lesser-known operators in C#](./2024/lesser-known-operators-in-cs.md)
* [Archiving bookmarks](./2024/bookmarks.md)
* [SHLVL PS1](./2023/shlvl.md)