Skip to content →

Category: Development

PyPI mirror support in Poetry

Background

Back in 2018 I wrote about adding support for PyPI mirrors to pipenv. Since then, I’ve switched to using Poetry for most of my Python projects.

In 2019, I opened python-poetry/poetry#1632 to propose PyPI mirror support in Poetry. The issue languished for a bit, and several attempts at addressing it were either abandoned or rejected. There were a couple of notable complications:

  • Since Poetry does not simply wrap pip, adding support for PyPI mirrors is more involved than just setting pip’s indexes.
  • In python-poetry/poetry#1632, users expressed differing expectations for the feature. Some desired a mechanism to replace arbitrary repositories at the global scope, whereas others were strictly interested in replacing the use of pypi.org.
  • Poetry’s maintainers wished to avoid introducing leaky abstractions.

Concerned about the suitability of a one-size-fits-all approach to mirror support, several participants in python-poetry/poetry#4944 discussed providing support through plugins, which were introduced in Poetry 1.2. In mid-October I released poetry-plugin-pypi-mirror, which focuses specifically on supporting mirrors of pypi.org.

Usage

To use poetry-plugin-pypi-mirror, it must first be installed — poetry self add poetry-plugin-pypi-mirror should be sufficient for most users. Advanced plugin installation instructions are provided in Poetry’s documentation.

To use a mirror, environment variable POETRY_PYPI_MIRROR_URL should be set to the appropriate URL.

For example:

POETRY_PYPI_MIRROR_URL=https://example.org/repository/pypi-proxy/simple/ poetry add pendulum

Or:

export POETRY_PYPI_MIRROR_URL=https://example.org/repository/pypi-proxy/simple/
poetry add cleo # uses mirror specified in first line
poetry lock     # also uses mirror specified in first line

Alternatively, POETRY_PYPI_MIRROR_URL can be set for the user, rather than before each use of poetry.

Leave a Comment

PyPI mirror support in pipenv

Note: I’ve since switched to poetry for Python dependency management.

As of June 25th, pipenv supports overriding PyPI urls with mirror indices. This is useful for developers who want to use a PyPI mirror or proxy repository with a pipenv-managed project, without the need to repeatedly modify the Pipfile to point to the mirror.

Usage

The --pypi-mirror parameter has been added to all commands which depend upon connectivity to PyPI.

If you’d like to override the default PyPI index urls with the url for a PyPI mirror, you can use the following:

$ pipenv install --pypi-mirror <mirror_url>

$ pipenv update --pypi-mirror <mirror_url>

$ pipenv sync --pypi-mirror <mirror_url>

$ pipenv lock --pypi-mirror <mirror_url>

$ pipenv uninstall --pypi-mirror <mirror_url>

Alternatively, you can set the PIPENV_PYPI_MIRROR environment variable.

Background

Since November 2017, pipenv has been the PyPA recommended dependency manager for Python projects. After using it for a bit, I noticed it lacked the ability to specify a PyPI mirror, a feature necessary for corporate adoption. In many corporate environments:

One Comment

Puppet module repository naming

Recently, I have been working on developing a few Puppet modules. When I started using puppet-lint to perform linting on my code, and puppet strings to generate documentation, I encountered related issues. To ensure that these issues were not specific to my project, I cloned Puppet’s ntp module, which is frequently used by the examples in Puppet’s documentation. I was able to reproduce both issues.

Leave a Comment