Commit 21baa07e authored by Craig Norris's avatar Craig Norris

Merge branch 'selhorn-pypi-edits' into 'master'

Docs: Edited PyPI topic

See merge request gitlab-org/gitlab!47022
parents a6b348b5 ed0a08e1
...@@ -184,7 +184,7 @@ to authenticate with the API: ...@@ -184,7 +184,7 @@ to authenticate with the API:
- [Maven Repository](../user/packages/maven_repository/index.md#authenticate-with-a-ci-job-token-in-maven) - [Maven Repository](../user/packages/maven_repository/index.md#authenticate-with-a-ci-job-token-in-maven)
- [NPM Repository](../user/packages/npm_registry/index.md#authenticate-with-a-ci-job-token) - [NPM Repository](../user/packages/npm_registry/index.md#authenticate-with-a-ci-job-token)
- [Nuget Repository](../user/packages/nuget_repository/index.md) - [Nuget Repository](../user/packages/nuget_repository/index.md)
- [PyPI Repository](../user/packages/pypi_repository/index.md#using-gitlab-ci-with-pypi-packages) - [PyPI Repository](../user/packages/pypi_repository/index.md#publish-a-pypi-package-by-using-cicd)
- [Generic packages](../user/packages/generic_packages/index.md#publish-a-generic-package-by-using-cicd) - [Generic packages](../user/packages/generic_packages/index.md#publish-a-generic-package-by-using-cicd)
- [Get job artifacts](job_artifacts.md#get-job-artifacts) - [Get job artifacts](job_artifacts.md#get-job-artifacts)
- [Pipeline triggers](pipeline_triggers.md) (using the `token=` parameter) - [Pipeline triggers](pipeline_triggers.md) (using the `token=` parameter)
......
...@@ -38,7 +38,7 @@ Learn more about using CI/CD to build: ...@@ -38,7 +38,7 @@ Learn more about using CI/CD to build:
- [Composer packages](../composer_repository/index.md#publish-a-composer-package-by-using-cicd) - [Composer packages](../composer_repository/index.md#publish-a-composer-package-by-using-cicd)
- [NuGet packages](../nuget_repository/index.md#publish-a-nuget-package-by-using-cicd) - [NuGet packages](../nuget_repository/index.md#publish-a-nuget-package-by-using-cicd)
- [Conan packages](../conan_repository/index.md#publish-a-conan-package-by-using-cicd) - [Conan packages](../conan_repository/index.md#publish-a-conan-package-by-using-cicd)
- [PyPI packages](../pypi_repository/index.md#using-gitlab-ci-with-pypi-packages) - [PyPI packages](../pypi_repository/index.md#publish-a-pypi-package-by-using-cicd)
- [Generic packages](../generic_packages/index.md#publish-a-generic-package-by-using-cicd) - [Generic packages](../generic_packages/index.md#publish-a-generic-package-by-using-cicd)
If you use CI/CD to build a package, extended activity information is displayed If you use CI/CD to build a package, extended activity information is displayed
......
...@@ -4,42 +4,28 @@ group: Package ...@@ -4,42 +4,28 @@ group: Package
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
--- ---
# GitLab PyPI Repository # PyPI packages in the Package Registry
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208747) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.10. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208747) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.10.
> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Core in 13.3. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Core in 13.3.
With the GitLab PyPI Repository, every project can have its own space to store PyPI packages. Publish PyPI packages in your project’s Package Registry. Then install the
packages whenever you need to use them as a dependency.
The GitLab PyPI Repository works with: The GitLab PyPI Repository works with:
- [pip](https://pypi.org/project/pip/) - [pip](https://pypi.org/project/pip/)
- [twine](https://pypi.org/project/twine/) - [twine](https://pypi.org/project/twine/)
## Setting up your development environment ## Build a PyPI package
You need a recent version of [pip](https://pypi.org/project/pip/) and [twine](https://pypi.org/project/twine/). This section covers creating a new example PyPI package to upload. This is a
quickstart to test out the **GitLab PyPI Registry**. If you already understand
## Enabling the PyPI Repository how to build and publish your own packages, move on to the [next section](#authenticate-with-the-package-registry).
NOTE: **Note:**
This option is available only if your GitLab administrator has
[enabled support for the Package Registry](../../../administration/packages/index.md).
After the PyPI Repository is enabled, it is available for all new projects
by default. To enable it for existing projects, or if you want to disable it:
1. Navigate to your project's **Settings > General > Visibility, project features, permissions**.
1. Find the Packages feature and enable or disable it.
1. Click on **Save changes** for the changes to take effect.
You should then be able to see the **Packages & Registries** section on the left sidebar.
## Getting started ### Install pip and twine
This section covers creating a new example PyPI package to upload. This is a You need a recent version of [pip](https://pypi.org/project/pip/) and [twine](https://pypi.org/project/twine/).
quickstart to test out the **GitLab PyPI Registry**. If you already understand how
to build and publish your own packages, move on to the [next section](#adding-the-gitlab-pypi-repository-as-a-source).
### Create a project ### Create a project
...@@ -97,8 +83,8 @@ Type "help", "copyright", "credits" or "license" for more information. ...@@ -97,8 +83,8 @@ Type "help", "copyright", "credits" or "license" for more information.
Hello from MyPyPiPackage Hello from MyPyPiPackage
``` ```
Once we've verified that the sample project is working as above, we can next After we've verified that the sample project is working as previously described,
work on creating a package. we can next work on creating a package.
### Create a package ### Create a package
...@@ -111,13 +97,13 @@ touch setup.py ...@@ -111,13 +97,13 @@ touch setup.py
This file contains all the information about our package. For more information This file contains all the information about our package. For more information
about this file, see [creating setup.py](https://packaging.python.org/tutorials/packaging-projects/#creating-setup-py). about this file, see [creating setup.py](https://packaging.python.org/tutorials/packaging-projects/#creating-setup-py).
GitLab identifies packages based on Becaue GitLab identifies packages based on
[Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names), [Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names),
so ensure your package name meets these requirements. ensure your package name meets these requirements. See the [installation section](#publish-a-pypi-package-by-using-cicd)
See the [installation section](#install-packages) for more details. for more details.
For this guide, we don't need to extensively fill out this file, simply add the For this guide, we don't need to extensively fill out this file. Add the
below to your `setup.py`: following to your `setup.py`:
```python ```python
import setuptools import setuptools
...@@ -138,7 +124,7 @@ setuptools.setup( ...@@ -138,7 +124,7 @@ setuptools.setup(
) )
``` ```
Save the file, then execute the setup like so: Save the file, and then execute the setup:
```shell ```shell
python3 setup.py sdist bdist_wheel python3 setup.py sdist bdist_wheel
...@@ -157,18 +143,21 @@ And confirm your output matches the below: ...@@ -157,18 +143,21 @@ And confirm your output matches the below:
mypypipackage-0.0.1-py3-none-any.whl mypypipackage-0.0.1.tar.gz mypypipackage-0.0.1-py3-none-any.whl mypypipackage-0.0.1.tar.gz
``` ```
Our package is now all set up and ready to be uploaded to the **GitLab PyPI The package is now all set up and is ready to be uploaded to the
Package Registry**. Before we do so, we next need to set up authentication. _GitLab PyPI Package Registry_. Before we do so, we next need to set up
authentication.
## Adding the GitLab PyPI Repository as a source ## Authenticate with the Package Registry
### Authenticating with a personal access token ### Authenticate with a personal access token
You need the following: You need the following:
- A personal access token. You can generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api` for repository authentication. - A personal access token. You can generate a
[personal access token](../../../user/profile/personal_access_tokens.md)
with the scope set to `api` for repository authentication.
- A suitable name for your source. - A suitable name for your source.
- Your project ID which can be found on the home page of your project. - Your project ID, which is found on the home page of your project.
Edit your `~/.pypirc` file and add the following: Edit your `~/.pypirc` file and add the following:
...@@ -183,13 +172,15 @@ username = __token__ ...@@ -183,13 +172,15 @@ username = __token__
password = <your personal access token> password = <your personal access token>
``` ```
### Authenticating with a deploy token ### Authenticate with a deploy token
You need the following: You need the following:
- A deploy token. You can generate a [deploy token](./../../project/deploy_tokens/index.md) with the `read_package_registry` and/or `write_package_registry` scopes for repository authentication. - A deploy token. You can generate a [deploy token](./../../project/deploy_tokens/index.md)
with the `read_package_registry` or `write_package_registry` scopes for
repository authentication.
- A suitable name for your source. - A suitable name for your source.
- Your project ID which can be found on the home page of your project. - Your project ID, which is found on the home page of your project.
Edit your `~/.pypirc` file and add the following: Edit your `~/.pypirc` file and add the following:
...@@ -204,16 +195,18 @@ username = <deploy token username> ...@@ -204,16 +195,18 @@ username = <deploy token username>
password = <deploy token> password = <deploy token>
``` ```
## Uploading packages ## Publish a PyPI package
When uploading packages, note that: When publishing packages, note that:
- The maximum allowed size is 50 Megabytes. - The maximum allowed size is 50 Megabytes.
- You cannot upload the same version of a package multiple times. If you try, you receive the error `Validation failed: File name has already been taken`. - You can't upload the same version of a package multiple times. If you try,
you'll receive the error `Validation failed: File name has already been taken`.
### Ensure your version string is valid ### Ensure your version string is valid
If your version string (for example, `0.0.1`) is invalid, it will be rejected. GitLab uses the following regex to validate the version string. If your version string (for example, `0.0.1`) is invalid, it will be rejected.
GitLab uses the following regex to validate the version string.
```ruby ```ruby
\A(?: \A(?:
...@@ -227,13 +220,14 @@ If your version string (for example, `0.0.1`) is invalid, it will be rejected. G ...@@ -227,13 +220,14 @@ If your version string (for example, `0.0.1`) is invalid, it will be rejected. G
)\z}xi )\z}xi
``` ```
You can play around with the regex and try your version strings on [this regular expression editor](https://rubular.com/r/FKM6d07ouoDaFV). You can experiment with the regex and try your version strings using this
[regular expression editor](https://rubular.com/r/FKM6d07ouoDaFV).
For more details about the regex used, please check the [documentation here](https://www.python.org/dev/peps/pep-0440/#appendix-b-parsing-version-strings-with-regular-expressions)) For more details about the regex used, review this [documentation](https://www.python.org/dev/peps/pep-0440/#appendix-b-parsing-version-strings-with-regular-expressions).
### Upload packages with Twine ### Publish a PyPI package by using twine
If you were following the guide above, then the `MyPyPiPackage` package should If you were following the steps on this page, the `MyPyPiPackage` package should
be ready to be uploaded. Run the following command: be ready to be uploaded. Run the following command:
```shell ```shell
...@@ -260,8 +254,8 @@ you can upload to the repository with the authentication inline: ...@@ -260,8 +254,8 @@ you can upload to the repository with the authentication inline:
TWINE_PASSWORD=<personal_access_token or deploy_token> TWINE_USERNAME=<username or deploy_token_username> python3 -m twine upload --repository-url https://gitlab.com/api/v4/projects/<project_id>/packages/pypi dist/* TWINE_PASSWORD=<personal_access_token or deploy_token> TWINE_USERNAME=<username or deploy_token_username> python3 -m twine upload --repository-url https://gitlab.com/api/v4/projects/<project_id>/packages/pypi dist/*
``` ```
If you did not follow the guide above, then you need to ensure your package If you didn't use the steps on this page, you need to ensure your package has
has been properly built and you [created a PyPI package with `setuptools`](https://packaging.python.org/tutorials/packaging-projects/). been properly built, and that you [created a PyPI package with `setuptools`](https://packaging.python.org/tutorials/packaging-projects/).
You can then upload your package using the following command: You can then upload your package using the following command:
...@@ -272,9 +266,42 @@ python -m twine upload --repository <source_name> dist/<package_file> ...@@ -272,9 +266,42 @@ python -m twine upload --repository <source_name> dist/<package_file>
Where: Where:
- `<package_file>` is your package filename, ending in `.tar.gz` or `.whl`. - `<package_file>` is your package filename, ending in `.tar.gz` or `.whl`.
- `<source_name>` is the [source name used during setup](#adding-the-gitlab-pypi-repository-as-a-source). - `<source_name>` is the [source name used during setup](#authenticate-with-the-package-registry).
### Publish a PyPI package by using CI/CD
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202012) in GitLab 13.4.
To work with PyPI commands within [GitLab CI/CD](./../../../ci/README.md), you
can use `CI_JOB_TOKEN` in place of the personal access token or deploy a token
in your commands.
For example:
```yaml
image: python:latest
## Install packages run:
script:
- pip install twine
- python setup.py sdist bdist_wheel
- TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token python -m twine upload --repository-url https://gitlab.com/api/v4/projects/${CI_PROJECT_ID}/packages/pypi dist/*
```
You can also use `CI_JOB_TOKEN` in a `~/.pypirc` file that you check into GitLab:
```ini
[distutils]
index-servers =
gitlab
[gitlab]
repository = https://gitlab.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/pypi
username = gitlab-ci-token
password = ${env.CI_JOB_TOKEN}
```
## Install a PyPI package
Install the latest version of a package using the following command: Install the latest version of a package using the following command:
...@@ -310,35 +337,3 @@ GitLab looks for packages using ...@@ -310,35 +337,3 @@ GitLab looks for packages using
so the characters `-`, `_`, and `.` are all treated the same and repeated characters are removed. so the characters `-`, `_`, and `.` are all treated the same and repeated characters are removed.
A `pip install` request for `my.package` looks for packages that match any of A `pip install` request for `my.package` looks for packages that match any of
the three characters, such as `my-package`, `my_package`, and `my....package`. the three characters, such as `my-package`, `my_package`, and `my....package`.
## Using GitLab CI with PyPI packages
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202012) in GitLab 13.4.
To work with PyPI commands within [GitLab CI/CD](./../../../ci/README.md), you can use
`CI_JOB_TOKEN` in place of the personal access token or deploy token in your commands.
For example:
```yaml
image: python:latest
run:
script:
- pip install twine
- python setup.py sdist bdist_wheel
- TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token python -m twine upload --repository-url https://gitlab.com/api/v4/projects/${CI_PROJECT_ID}/packages/pypi dist/*
```
You can also use `CI_JOB_TOKEN` in a `~/.pypirc` file that you check into GitLab:
```ini
[distutils]
index-servers =
gitlab
[gitlab]
repository = https://gitlab.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/pypi
username = gitlab-ci-token
password = ${env.CI_JOB_TOKEN}
```
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment