Integrating with Thoth¶
Check the following Tutorial to get hands-on experience on how to use Thoth and what Thoth can provide to you.
Project Thoth can give advises to your software stacks. To consume Thoth’s recommendations, there are multiple ways on how to integrate:
Command line interface - Thamos CLI
Kebechet (GitHub application)
OpenShift s2i build process
Thamos library (not fully supported yet as API might change)
Pre-requirements for your project¶
See `thoth-station/cli-examples repository <https://github.com/thoth-station/cli-examples/>`__ as an example on how to structure your project and use Thoth.
To let Thoth manage your Python application, your application has to use
Pipenv to manage virtual environment and
application dependencies. Pipenv’s two main files,
Pipfile.lock, has to be placed in the root of your Python application
If you use
requirements.txt, the easiest way how to convert to Pipenv’s
Pipfile.lock is to run the following command:
pipenv install --requirements requirements.txt
And add both file produced,
Pipfile.lock, into your Git
It’s also possible to use pip
format as well as format used by
pip-tools. To use these formats,
you will need to adjust
requirements_format configuration option in your
.thoth.yaml configuration file.
It’s recommended to use Pipenv files if possible. Pipenv introduces more consistent files that track Python package indexes used as well as artifact hashes in the lock file explicitly.
By switching to
pip-compile file format the behaviour of file lookup
is following (sorted based on priority):
requirements.infiles are present,
requirements.txtfile is used as a lockfile and
requirements.instates direct dependencies (
requirements.infile is present, it is used as a file containing direct dependencies (
requirements.txtfile is present, it is used as a file containing direct dependencies (raw
Thoth’s output of
requirements.txt uses implicitly hashes of artifacts -
pip-compile --generate-hashes. It is required to state artifact
requirements.txt is treated as a lockfile.
Command Line Interface - Thamos CLI¶
The easiest way how to get recommendations from Thoth service is to install Thamos (Thoth’s CLI and library):
pip3 install -U thamos
And configure your project to use Thoth’s recommendations and ask for them:
cd your/project/path thamos config thamos advise
Before you ask for an advise, make sure your Pipenv’s files
Pipfile.lock are present in the root directory of your project
or respecting overlays directory configuration:
ls -la your/project/path .. .thoth.yaml .. Pipfile Pipfile.lock ..
Once Thoth responds back with recommendations, you can install your dependencies using:
thamos install --deploy --dev # Or directly: # thamos advise --install --dev
OpenShift Python s2i build process¶
Thoth can be used in OpenShift’s s2i process where it can produce recommendations targeting your specific hardware configuration you use to run your application inside the cluster (e.g. specific GPU available in the cluster).
Check thoth-station/s2i-example-migration with a step-by-step tutorial on how to port an already existing Python s2i application to Thoth.
You can find a list of base images which you can use with Thoth in s2i-thoth repository with detailed instructions on how to use Thoth in the OpenShift’s s2i process. The container images are hosted at quay.io/organization/thoth-station.
You can use a tool called thoth-s2i that can automatically migrate your existing s2i application to use Thoth.
Thoth’s s2i container images can be configured using environment variables supplied to the build config. Follow thoth-station/s2i-thoth repository with all the required instructions to setup OpenShift S2I. See also configuration options for Thoth’s client present in Thamos documentation that apply in the build process as it uses Thamos under the hood.
To see available S2I runtime environments for which backend can give you advises, issue:
An example of an S2I application powered by Thoth S2I can be found in thoth-station/s2i-example repository.
Kebechet - GitHub application¶
Here is are few easy steps describing how you can add Kebechet to your GitHub project:
Install Thamos CLI tool:
pip3 install thamos # keep in mind: requires Python 3.6+!!
Go to the repository that should be managed by Thoth which already has
Setup Thamos configuration:
That’s it - thamos would create a “
.thoth.yaml” file for you. This file can
be added to your Git repository and GitHub application can take care of your
Next, you can adjust managers you want to have enabled on your repository.
Here, for example, we want
update manager to run on the
demo repo so the configuration is:
managers: - name: update configuration: labels: [bot, kebechet] - name: version configuration: labels: [bot, kebechet] changelog_file: true
Kebechet cares about the managers you add under the manager section. You will find how to define the manager config under each of the manager documentation - kebechet/managers.
Now that you are done with the setup of which managers you want to be run on your project, you are done with the major part. We would next install the GitHub app to ensure we receive webhooks from the repository, please install Khebhut, which is an alias for Kebechet. That’s it, Kebechet is now ready to maintain your Python project.
Container image build analyses¶
To help us improving recommendations, you can use integrations with container image build systems that can report information about builds to Thoth to improve recommendations. Simply, we aggregate information about build failures, learn from them and improve the recommendation engine so that it will provide you a Python stack that can be assembled.
If you use OpenShift builds, you can install a component called build-watcher which will send us relevant information so that we can improve Thoth’s recommendation engine. Follow instructions that can be found in thoth-station/build-watcher repository for more info.
When using AICoE-CI, build information are automatically sent to Thoth backend.
Follow documentation in thoth-station/jupyterlab-requirements repository for more info.