From ac1128a0d1a14d7a0ebfb9075ea6ee662242c1ac Mon Sep 17 00:00:00 2001 From: "Jason R. Coombs" Date: Thu, 22 Oct 2026 11:53:10 -0700 Subject: [PATCH 1/4] gh-123976: Document backport caveats for custom providers Describe how the separate class hierarchies in importlib.metadata and the importlib_metadata backport affect custom providers: which objects the backport substitutes, which it passes through with a warning, and why a provider should implement the whole interface and rely on the shared protocols rather than on a particular base class. Co-Authored-By: Claude Opus 5 --- Doc/library/importlib.metadata.rst | 50 +++++++++++++++++++ ...-10-22-18-51-05.gh-issue-123976.GKLjFT.rst | 2 + 2 files changed, 52 insertions(+) create mode 100644 Misc/NEWS.d/next/Documentation/2026-10-22-18-51-05.gh-issue-123976.GKLjFT.rst diff --git a/Doc/library/importlib.metadata.rst b/Doc/library/importlib.metadata.rst index 1378a131f0ce651..95d05bbc4cde3f3 100644 --- a/Doc/library/importlib.metadata.rst +++ b/Doc/library/importlib.metadata.rst @@ -735,3 +735,53 @@ packages served by the ``DatabaseImporter``, assuming that the The ``DatabaseDistribution`` may also provide other metadata files, like ``RECORD`` (required for :attr:`!Distribution.files`) or override the implementation of :attr:`!Distribution.files`. See the source for more inspiration. + +Because ``DatabaseDistribution`` derives from :class:`Distribution` as defined +here, consumers that use the :pypi:`importlib_metadata` backport instead of +this module receive objects from a hierarchy they do not expect. See +:ref:`metadata-backport-caveats`. + +.. _metadata-backport-caveats: + +Caveats for backports +--------------------- + +The :pypi:`importlib_metadata` package backports this module and supplies its +own :class:`!Distribution`, :class:`!PackagePath`, and metadata classes. Those +classes are separate from the ones in :mod:`!importlib.metadata`; neither set +derives from the other. + +A custom provider has no say in which of the two a consumer uses. Both +libraries discover distributions by querying the finders on +:data:`sys.meta_path`, so a single provider installed there serves both. A +consumer calling ``importlib_metadata.distributions()`` expects that package's +objects, but a provider deriving from the :class:`Distribution` class +documented here supplies objects from the other hierarchy. + +The backport settles most of this on the consumer's behalf, substituting its +own class for each object it recognizes, such as a :class:`!PathDistribution`, +a :class:`PackagePath`, or an :class:`email.message.Message`. It cannot do so +for an arbitrary :class:`Distribution` subclass from a custom provider, having +no way to reconstruct that subclass. Such an object is passed through +unchanged, and the backport emits a :exc:`UserWarning` reporting the +unrecognized subclass. + +A passed-through object still serves the consumer, provided it implements the +whole interface rather than part of it. So, when writing a custom provider: + +* Supply objects that present the whole interface, not part of it. Deriving + from :class:`Distribution`, as the example above does, achieves this: the + base class builds the rest of the interface on + :meth:`~Distribution.read_text` and :meth:`~Distribution.locate_file`. A + provider that assembles its objects some other way supplies every member + itself. +* Where the provider itself consumes metadata, and in the guidance offered to + its users, rely on the object satisfying that interface rather than on its + type. The backport publishes ``IDistribution``, ``IPackagePath``, and + ``PackageMetadata`` protocols for this purpose, which both its classes and + the ones documented here satisfy. + +Deriving from whichever of the two libraries happens to be importable is not a +general answer. A provider cannot know whether a consumer will import the +backport, and importing it to find out activates it in environments that would +otherwise never have used it. diff --git a/Misc/NEWS.d/next/Documentation/2026-10-22-18-51-05.gh-issue-123976.GKLjFT.rst b/Misc/NEWS.d/next/Documentation/2026-10-22-18-51-05.gh-issue-123976.GKLjFT.rst new file mode 100644 index 000000000000000..14f6adcda1996c6 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2026-10-22-18-51-05.gh-issue-123976.GKLjFT.rst @@ -0,0 +1,2 @@ +Document how custom ``importlib.metadata`` providers interact with the +``importlib_metadata`` backport. From ab57ac7aa45fae3217f77f1b530401830a8a4d9d Mon Sep 17 00:00:00 2001 From: "Jason R. Coombs" Date: Thu, 22 Oct 2026 14:40:59 -0700 Subject: [PATCH 2/4] gh-123976: Warn only on an incomplete provider importlib_metadata now decides the warning by conformance to IDistribution rather than by class identity, so a provider that presents the whole interface draws no warning and the warning names what a class omits. Co-Authored-By: Claude Opus 5 --- Doc/library/importlib.metadata.rst | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/Doc/library/importlib.metadata.rst b/Doc/library/importlib.metadata.rst index 95d05bbc4cde3f3..6184df1760b7384 100644 --- a/Doc/library/importlib.metadata.rst +++ b/Doc/library/importlib.metadata.rst @@ -760,21 +760,21 @@ documented here supplies objects from the other hierarchy. The backport settles most of this on the consumer's behalf, substituting its own class for each object it recognizes, such as a :class:`!PathDistribution`, -a :class:`PackagePath`, or an :class:`email.message.Message`. It cannot do so -for an arbitrary :class:`Distribution` subclass from a custom provider, having -no way to reconstruct that subclass. Such an object is passed through -unchanged, and the backport emits a :exc:`UserWarning` reporting the -unrecognized subclass. +a :class:`PackagePath`, or an :class:`email.message.Message`. It has no way to +reconstruct an arbitrary :class:`Distribution` subclass from a custom +provider, so it passes such an object through unchanged. That serves the +consumer as long as the object presents the whole interface; where the class +omits a member the interface requires, the backport emits a +:exc:`UserWarning` naming what is missing. -A passed-through object still serves the consumer, provided it implements the -whole interface rather than part of it. So, when writing a custom provider: +So, when writing a custom provider: * Supply objects that present the whole interface, not part of it. Deriving from :class:`Distribution`, as the example above does, achieves this: the base class builds the rest of the interface on - :meth:`~Distribution.read_text` and :meth:`~Distribution.locate_file`. A - provider that assembles its objects some other way supplies every member - itself. + :meth:`~Distribution.read_text` and :meth:`~Distribution.locate_file`, and + such a provider draws no warning. A provider that assembles its objects + some other way supplies every member itself. * Where the provider itself consumes metadata, and in the guidance offered to its users, rely on the object satisfying that interface rather than on its type. The backport publishes ``IDistribution``, ``IPackagePath``, and From 69a4d4841fde358a1c68bd3592acb90a72cf9a80 Mon Sep 17 00:00:00 2001 From: "Jason R. Coombs" Date: Thu, 22 Oct 2026 14:51:41 -0700 Subject: [PATCH 3/4] gh-123976: Separate the consumer and provider caveats The guidance to rely on the interface rather than the type addresses every consumer, not only a provider that consumes metadata, so promote the caveats to their own section and split it by audience. Note that the protocols come from the backport, which this module does not yet present. Co-Authored-By: Claude Opus 5 --- Doc/library/importlib.metadata.rst | 75 ++++++++++++++++-------------- 1 file changed, 39 insertions(+), 36 deletions(-) diff --git a/Doc/library/importlib.metadata.rst b/Doc/library/importlib.metadata.rst index 6184df1760b7384..5c6d6b3a91f2b97 100644 --- a/Doc/library/importlib.metadata.rst +++ b/Doc/library/importlib.metadata.rst @@ -736,50 +736,53 @@ The ``DatabaseDistribution`` may also provide other metadata files, like ``RECORD`` (required for :attr:`!Distribution.files`) or override the implementation of :attr:`!Distribution.files`. See the source for more inspiration. -Because ``DatabaseDistribution`` derives from :class:`Distribution` as defined -here, consumers that use the :pypi:`importlib_metadata` backport instead of -this module receive objects from a hierarchy they do not expect. See -:ref:`metadata-backport-caveats`. +``DatabaseDistribution`` derives from :class:`Distribution` as defined here, +which has consequences for consumers that use the :pypi:`importlib_metadata` +backport instead of this module. See :ref:`metadata-backport-providers`. + .. _metadata-backport-caveats: Caveats for backports ---------------------- +===================== The :pypi:`importlib_metadata` package backports this module and supplies its own :class:`!Distribution`, :class:`!PackagePath`, and metadata classes. Those classes are separate from the ones in :mod:`!importlib.metadata`; neither set -derives from the other. - -A custom provider has no say in which of the two a consumer uses. Both -libraries discover distributions by querying the finders on -:data:`sys.meta_path`, so a single provider installed there serves both. A -consumer calling ``importlib_metadata.distributions()`` expects that package's -objects, but a provider deriving from the :class:`Distribution` class -documented here supplies objects from the other hierarchy. - -The backport settles most of this on the consumer's behalf, substituting its -own class for each object it recognizes, such as a :class:`!PathDistribution`, -a :class:`PackagePath`, or an :class:`email.message.Message`. It has no way to -reconstruct an arbitrary :class:`Distribution` subclass from a custom -provider, so it passes such an object through unchanged. That serves the -consumer as long as the object presents the whole interface; where the class -omits a member the interface requires, the backport emits a -:exc:`UserWarning` naming what is missing. - -So, when writing a custom provider: - -* Supply objects that present the whole interface, not part of it. Deriving - from :class:`Distribution`, as the example above does, achieves this: the - base class builds the rest of the interface on - :meth:`~Distribution.read_text` and :meth:`~Distribution.locate_file`, and - such a provider draws no warning. A provider that assembles its objects - some other way supplies every member itself. -* Where the provider itself consumes metadata, and in the guidance offered to - its users, rely on the object satisfying that interface rather than on its - type. The backport publishes ``IDistribution``, ``IPackagePath``, and - ``PackageMetadata`` protocols for this purpose, which both its classes and - the ones documented here satisfy. +derives from the other. Both libraries discover distributions by querying the +finders on :data:`sys.meta_path`, so one custom provider serves consumers of +both and has no say in which of the two a given consumer uses. + +For consumers +------------- + +A consumer calling ``importlib_metadata.distributions()`` expects that +package's objects, but a custom provider may supply objects from the hierarchy +documented here. The backport settles most of this on the consumer's behalf, +substituting its own class for each object it recognizes, such as a +:class:`!PathDistribution`, a :class:`PackagePath`, or an +:class:`email.message.Message`. It has no way to reconstruct an arbitrary +:class:`Distribution` subclass from a custom provider, so it passes such an +object through unchanged. + +Rely on an object satisfying the interface rather than on its type, and do not +test it with :func:`isinstance` against a concrete class. The backport +publishes ``IDistribution``, ``IPackagePath``, and ``PackageMetadata`` +protocols for this purpose, which the classes of both libraries satisfy. +Import them from the backport; this module does not present them yet. + +.. _metadata-backport-providers: + +For custom providers +-------------------- + +A provider's objects reach consumers of both libraries, so supply objects that +present the whole interface, not part of it. Deriving from +:class:`Distribution`, as the example above does, achieves this: the base +class builds the rest of the interface on :meth:`~Distribution.read_text` and +:meth:`~Distribution.locate_file`. Where a class omits a member the interface +requires, the backport emits a :exc:`UserWarning` naming what is missing; a +provider presenting the whole interface draws no warning. Deriving from whichever of the two libraries happens to be importable is not a general answer. A provider cannot know whether a consumer will import the From be9a5dede19969f65fc41e718d6c493e206c019b Mon Sep 17 00:00:00 2001 From: "Jason R. Coombs" Date: Thu, 22 Oct 2026 14:56:02 -0700 Subject: [PATCH 4/4] gh-123976: Document the protocols shared with the backport Assume the shared protocols from python/importlib_metadata#505 are presented by this module as well: document IDistribution and IPackagePath, correct PackageMetadata, which is already a protocol here rather than an implementation of the backport's, and write the caveats around the protocols as the interface both libraries satisfy. Co-Authored-By: Claude Opus 5 --- Doc/library/importlib.metadata.rst | 55 +++++++++++++------ ...-10-22-18-51-05.gh-issue-123976.GKLjFT.rst | 5 +- 2 files changed, 42 insertions(+), 18 deletions(-) diff --git a/Doc/library/importlib.metadata.rst b/Doc/library/importlib.metadata.rst index 5c6d6b3a91f2b97..3ab8505e1e9ebc8 100644 --- a/Doc/library/importlib.metadata.rst +++ b/Doc/library/importlib.metadata.rst @@ -237,11 +237,12 @@ Distribution metadata .. class:: PackageMetadata - A concrete implementation of the - `PackageMetadata protocol `_. + A protocol describing the interface distribution metadata presents. The + :pypi:`importlib_metadata` backport presents the same protocol; see + :ref:`metadata-backport-caveats`. - In addition to providing the defined protocol methods and attributes, subscripting - the instance is equivalent to calling the :meth:`!get` method. + In addition to the protocol methods and attributes, subscripting an + instance is equivalent to calling the :meth:`!get` method. Every `Distribution Package `_ includes some metadata, which you can extract using the :func:`!metadata` function:: @@ -332,6 +333,13 @@ Distribution files check if it ``exists()``, to traverse using ``joinpath()`` and ``parent``, and to retrieve data using ``read_text()`` and ``read_bytes()``. +.. class:: IPackagePath + + A protocol describing the interface a :class:`PackagePath` presents. See + :ref:`metadata-backport-caveats`. + + .. versionadded:: 3.16 + The :func:`!files` function takes a `Distribution Package `_ name and returns all of the files installed by this distribution. For example:: @@ -544,6 +552,17 @@ The same applies for :func:`entry_points` and :func:`files`. A shortcut for ``distribution.locate_file(filename).read_text()``. +.. class:: IDistribution + + A protocol describing the interface a :class:`Distribution` presents. + + A consumer that targets this protocol, rather than a concrete class, works + with distributions from either this module or the + :pypi:`importlib_metadata` backport, however they were supplied. See + :ref:`metadata-backport-caveats`. + + .. versionadded:: 3.16 + .. _distribution-discovery: Distribution Discovery @@ -748,10 +767,15 @@ Caveats for backports The :pypi:`importlib_metadata` package backports this module and supplies its own :class:`!Distribution`, :class:`!PackagePath`, and metadata classes. Those -classes are separate from the ones in :mod:`!importlib.metadata`; neither set -derives from the other. Both libraries discover distributions by querying the -finders on :data:`sys.meta_path`, so one custom provider serves consumers of -both and has no say in which of the two a given consumer uses. +classes are separate from the ones here; neither set derives from the other. +Both libraries discover distributions by querying the finders on +:data:`sys.meta_path`, so one custom provider serves consumers of both and has +no say in which of the two a given consumer uses. + +Both libraries present the same protocols, :class:`IDistribution`, +:class:`IPackagePath`, and :class:`PackageMetadata`, and the classes of both +satisfy them. Those protocols, rather than any concrete class, are the +interface to write against. For consumers ------------- @@ -765,11 +789,10 @@ substituting its own class for each object it recognizes, such as a :class:`Distribution` subclass from a custom provider, so it passes such an object through unchanged. -Rely on an object satisfying the interface rather than on its type, and do not -test it with :func:`isinstance` against a concrete class. The backport -publishes ``IDistribution``, ``IPackagePath``, and ``PackageMetadata`` -protocols for this purpose, which the classes of both libraries satisfy. -Import them from the backport; this module does not present them yet. +Annotate against the protocols, then, and do not test an object with +:func:`isinstance` against a concrete class. Code that needs the protocols on +earlier Python versions can import them from the backport, which presents them +there too. .. _metadata-backport-providers: @@ -777,12 +800,12 @@ For custom providers -------------------- A provider's objects reach consumers of both libraries, so supply objects that -present the whole interface, not part of it. Deriving from +satisfy :class:`IDistribution` whole, not in part. Deriving from :class:`Distribution`, as the example above does, achieves this: the base class builds the rest of the interface on :meth:`~Distribution.read_text` and -:meth:`~Distribution.locate_file`. Where a class omits a member the interface +:meth:`~Distribution.locate_file`. Where a class omits a member the protocol requires, the backport emits a :exc:`UserWarning` naming what is missing; a -provider presenting the whole interface draws no warning. +provider satisfying the protocol whole draws no warning. Deriving from whichever of the two libraries happens to be importable is not a general answer. A provider cannot know whether a consumer will import the diff --git a/Misc/NEWS.d/next/Documentation/2026-10-22-18-51-05.gh-issue-123976.GKLjFT.rst b/Misc/NEWS.d/next/Documentation/2026-10-22-18-51-05.gh-issue-123976.GKLjFT.rst index 14f6adcda1996c6..614d2c6595c3246 100644 --- a/Misc/NEWS.d/next/Documentation/2026-10-22-18-51-05.gh-issue-123976.GKLjFT.rst +++ b/Misc/NEWS.d/next/Documentation/2026-10-22-18-51-05.gh-issue-123976.GKLjFT.rst @@ -1,2 +1,3 @@ -Document how custom ``importlib.metadata`` providers interact with the -``importlib_metadata`` backport. +Document the ``IDistribution``, ``IPackagePath``, and ``PackageMetadata`` +protocols shared with the ``importlib_metadata`` backport, and how custom +providers and consumers of ``importlib.metadata`` should use them.