Repository navigation
Encourage using feature detection instead of distro name/version #221
Description
Activity
completely agreed with that. Use it carefully if you need to. But better try to make decisions on features found.
While I definitely agree we should add a warning, I disagree that we should state that it shouldn't be used to make decisions. Rather, I think it makes sense to state that "It should be used to make decisions only if it makes sense".
A for instance:
Cloudify uses Wagon to build their plugin packages. Packages that contain C extensions are compiled for a specific distribution and is expected to run on that distribution and only if the distribution contains the exact glibc (etc) as the build environment.
You could say that they should start checking ABI's, features, glibc versions, etc.. for every possible distribution out there. What made more sense to them, is to check whether the destination distribution's id and version are the same as the distro the package was build on and document the culprits. Someone else might decide otherwise, but that doesn't mean that it's a bad decision.What I do agree with, is that "Using the distribution info to decide on things reduces precision. If you care about precision, go for a lower-level decision-making algorithm".
WDYT?
What made more sense to them, is to check whether the destination distribution's id and version are the same as the distro the package was build on and document the culprits. Someone else might decide otherwise, but that doesn't mean that it's a bad decision.
That is exactly what forces Ubuntu derivatives to identify as Ubuntu (and optionally publishing their “true name” in a non-standard location, which will in turn force
distroto also consider that non-standard location, making it de-facto standard, and so on).The particular derivative case is solved by Debian by using
dpkg-vendor --derives-from. So a conditional check would be "Is this Ubuntu or derived from Ubuntu?" instead of "Is this Ubuntu?"That looks like a distro-specific solution, which requires a special case in the detection code 🙁
I just mentioned it as an example of how the problem is solved today. I didn't intend to suggest that you used that as-is. But
os-releasedoes haveID_LIKE, which provides the information you need to implement something similar. I think you could implement a "am I like X" function using that data.Reacted by Samuel FORESTIERSorry for the delay.
Would the following work for the documentation, under Overview and motivation?Preferring feature detection
Information from this package is useful for logging and problem reporting.
However, it is usually a bad idea to base automated decisions on the
information.
Instead, try to use feature detection. If you need to install packages,
look foryumvs.apt-getcommands, if you need a specific version of
glibc, query the library version directly, and so on.If people use distro to apply custom workarounds for, say, Ubuntu,
less popular derivatives like Linux Mint will need to report themselves as
Ubuntu to get those workarounds. That then leads to the need for more powerful
distro reporting tools, which then leads to more distro-specific workarounds,
and more distros needing to pose as other distros to get the workarounds.
Please, do not feed the vicious circle.Reacted by basak[...] But
os-releasedoes haveID_LIKE, which provides the information you need to implement something similar. I think you could implement a "am I like X" function using that data.But what does
ID_LIKEmean? In what respect is the distro like another distro?
If I take Ubuntu, re-package it all into RPM, and put inyuminstead ofapt-get, should I label it as “like Ubuntu”?I believe this scope problem is also the main reason for removing
platform.linux_distribution()from Python: We don't know the questions we are trying to answer. Or, we're trying to give a single answer to too many questions at once.- On Fri, Apr 12, 2019 at 07:30:32AM -0700, Petr Viktorin wrote: @basak > [...] But `os-release` does have `ID_LIKE`, which provides the information you need to implement something similar. I think you could implement a "am I like X" function using that data. But what does `ID_LIKE` mean? In what respect is the distro *like* another distro? If I take Ubuntu, re-package it all into RPM, and put in `yum` instead of `apt-get`, should I label it as “like Ubuntu”?I don't think it needs to specify crazy situations and cover all cases before it can be useful. If you're going to make such invasive changes in your derivative, then you'll need to patch things all over the place to accomodate differing behaviour anyway. At that point, you might as well add specific behaviour for the new scenario in those patches anyway (and send them upstream etc). In the majority of cases, something like "like Debian" will do, and automatically incorporate Ubuntu and Mint. FWIW, the specification of ID_LIKE is in here: https://www.freedesktop.org/software/systemd/man/os-release.htmlI believe this scope problem is also the main reason for removing `platform.linux_distribution()` from Python: We don't know the questions we are trying to answer. Or, we're trying to give a single answer to too many questions at once.Related are my comments and their replies at https://bugs.python.org/issue28167. This isn't a problem that the Python world needs to solve alone. The Free Software community has converged on os-release, and it would be very useful to at least pass that through in a Pythonic way. Extra smarts is fine, but it only really needs to be added on as an opt-in now, or used where (presumably only obscure distributions now when) os-release isn't available. I'm not suggesting that feature detection is wrong; it certainly is the most granular and future-proof way of solving the general problem. I'm only saying that it can be difficult to do right in some cases, and in those cases would be overkill when ID_LIKE would suffice - if only there were a library that allowed Python developers to do that test easily in one line.
[...] I'm not suggesting that feature detection is wrong; it certainly is the most granular and future-proof way of solving the general problem. I'm only saying that it can be difficult to do right in some cases, and in those cases would be overkill when ID_LIKE would suffice - if only there were a library that allowed Python developers to do that test easily in one line.
Oh, nothing against quick hacks! But I'd like to make sure people know what the proper solution is.
Instead of having my new Linux masquerade as something it's not, I should be able to send a PR to the offending project to enable feature detection, and link todistro's official docs to support my case :)I'm the author of
extra-platforms, a library that addresses several of the concerns raised in this thread, particularly around platform grouping and the "browser sniffing" problem.extra-platformstakes a different approach fromdistro: instead of returning raw strings that callers must compare against, it provides structured traits (individual platforms) and groups (families of platforms), with set-like operations. This means you write code against families rather than individual distro names.For example, instead of:
distro_id = distro.id() if distro_id in ("debian", "ubuntu", "linuxmint", "raspbian", "kali", ...): subprocess.run(["apt-get", "install", package])
You write:
from extra_platforms import LINUX, BSD, SYSTEM_V, current_platform if current_platform() in LINUX: # Any Linux distribution ...
The library ships with pre-defined groups (
LINUX,BSD,SYSTEM_V,LINUX_LIKE,UNIX, etc.) and auto-generatesis_<group>()convenience functions:from extra_platforms import is_linux, is_bsd, is_unix if is_linux(): ...
Groups support set operations, so you can compose ad-hoc families:
from extra_platforms import LINUX, DEBIAN, UBUNTU non_debian_linux = LINUX - {DEBIAN, UBUNTU}
It also provides pytest decorators for conditional test execution:
from extra_platforms.pytest import skip_linux, unless_bsd @unless_bsd def test_bsd_only_feature(): ...
This addresses @basak's point about wanting a one-liner for "is this like Debian?" without requiring callers to maintain their own list of derivative names. And it partially addresses @encukou's concern: by coding against groups rather than individual distro IDs, you avoid the worst of the "browser sniffing" pattern. Adding a new Debian derivative to the
LINUXgroup in the library means all downstream consumers pick it up automatically, rather than each project needing to update its own string comparison list.I agree with @encukou that feature detection should be preferred when possible. But for the cases where you genuinely need to reason about platform families (conditional dependencies, CI matrix configuration, test skipping), having a structured grouping system is better than ad-hoc string matching.
Reacted by Petr Viktorin
Hello! I'm the person responsible for dropping
platform.linux_distrubutionfrom Python 3.8. We're now recommendingdistroinstead. Thank you for all the work going into this module – it's certainly useful in certain cases, like finding out what to put in a log or problem report.However, think that
distro's docs should include a warning against using it for decisions, because a platform's name is not a good indicator of what is possible on it.Once people apply custom workarounds for Ubuntu (and use distro info for that), less popular derivatives like Linux Mint will need to report themselves as Ubuntu to get those workarounds. That then leads to the need for better distro reporting tools, like this one... which then leads to more distro-specific workarounds. It's a vicious circle.
It's the same problem as "browser sniffing" in the web world, where modern “user agent” strings contain mostly lies:
You'll find warnings against using browser-agent in any respectable recent documentation on the topic.