Need requirements.txt with fixed package versions

[pohlt]

Here's my use case: I want to build a Python project inside a container with exactly the package versions as in pdm.lock, but without installing pdm itself, but using a pip install -r requirements.txt in the container instead.

Just to be clear: In pyproject.toml I specify versions using ~= to only use compatible versions, but in the container I want to use exactly the same package versions as mentioned in pdm.lock-

• I tried pdm export --pyproject, but that doesn't give me the fixed versions as in pdm.lock.
• I tried pdm export which gives me fixed versions, but requires an installed Python versions as mentioned in pyproject.toml, which might not be available on the host system (but would be in the container).

I'm sure I'm missing something, but why can't I just export the packages from the lock file (with any Python 3.x interpreter)?

[pawamoy]

pdm export -f requirements?

[pohlt]

Isn't that the default? I don't see a difference.

And it still fails if the correct Python version (as requested in requires-python) is missing.

[pawamoy]

Right, my bad, didn't know this was the default. Then I don't know!

Note that you can use a multi-stage build to install things with PDM, without keeping PDM in the final image: https://pdm-project.org/latest/usage/advanced/#use-pdm-in-a-multi-stage-dockerfile

[pohlt]

Thanks for the hint. The multi-stage build is exactly what I'm doing right now as a workaround.

But it feels kind of overkill to install pdm inside the container just to install packages where pdm already did the heavy-lifting of locking on the host and a pip would be good enough if I had a requirements.txt with all versions fixed.

[frostming]

If you are looking for a requirements file that would work in a certain range of python versions., in short words, we don't support.
The first line in the output of the export command also indicates this fact.

But I am a bit confused by what you said:

• but requires an installed Python versions as mentioned in pyproject.toml, which might not be available on the host system (but would be in the container).

And it still fails if the correct Python version (as requested in requires-python) is missing.

Are you trying to use a Python version that is NOT in the range of requires-python? That apparently won't work. requires-python is a field to restrict what python versions can be used, and should be respected at any time.

[pohlt]

Obviously, I did a bad job explaining my use case. Here's another try.

I have

• a host machine with some Python (e.g. 3.7) installed,
• a repo which includes some Python project, a ´pyproject.toml(requiring Python 3.12) and apdm.lock` file.

I now want to build a container for this Python project. Of course, this container includes the required Python 3.12.

For buidling the container, I want to create a requirements.txt file with fixed versions of all required packages. I don't have (and don't want) Python 3.12 to be installed on the host system. In the container, I want to run a simple pip install -r requirements.txt to install all required packages.

How can I use some flavor of pdm export to

• create the described requirements.txt with fixed versions
• using only the Python 3.7 (and pdm) which is installed on the host system?

[frostming]

Do you mean to lock dependencies for Python 3.12 on Python 3.7? You can't.

We use the target_python below to evaluate package links:

pdm/src/pdm/environments/base.py

Lines 71 to 77 in 70f4985

	@cached_property
	def target_python(self) -> unearth.TargetPython:
	from unearth import TargetPython
	python_version = self.interpreter.version_tuple
	python_abi_tag = get_python_abi_tag(str(self.interpreter.executable))
	return TargetPython(python_version, [python_abi_tag])

We don't support providing python_version from outside. It is a value derived from the python used by your project.

[pohlt]

Isn't all the required information to generate a requirements.txt file (what package do I need in which version) contained in pyproject.toml and pdm.lock?

[frostming]

No, the final set of packages to be installed may vary depending on which Python version are being used, think of a foo; python_version >= '3.8' dependency, but unfortunately, pdm export lacks the ability to record and possibly merge all those markers to form a static requirements.txt. What it does is to read the current python used when running export and tailor the dependency list to get a requirements.txt. This is because pdm.lock may contain too many packages not needed by the target environment. For example, a lockfile may look like the following(translated to requirements.txt format):

foo==1; python_version<'3.11'
bar==1; python_version<'3.10'
     # bar depends on
     foo; python_version>='3.9'
baz==1; python_version>='3.11'
     # baz depends on
     foo

In this lockfile, what should the requirements.txt look like that would work for python 3.8, 3.9, 3.10 and 3.11? Especially, what is the environment marker for foo, note that the dependency list shouldn't appear in the result since the version isn't pinned.

It would look like:

foo==1;  # ???
bar==1; python_version<'3.10'
baz==1; python_version>='3.11'

If you see this and think that we can merge markers from different sources, then you have found the right path! But that is not as easy as you might think although it is in our future plans. So now PDM chooses to only tailor the package list for the current python version.

[frostming]

And this kind of dependency is not uncommon, such as importlib-metadata, tomlli, typing_extensions

[pohlt]

Ok, got it. Thanks for the explanation.

I'm going stick to the multi-stage build then.