Package Combinations Predictor¶
Check high level predictor docs for predictor basics.
Yet another predictor was designed for Dependency Monkey runs and is suitable for generating combinations of certain packages in a software stack. An example use case can be a TensorFlow stack with all the packages locked to a specific version except for NumPy which should be tested in different versions. This predictor can also work when multiple combinations should be consider - e.g. NumPy with all the possible absl-py packages.
When computing combinations of certain package or packages it is often a good practice to pin versions of all the other packages to specific versions. This can be accomplished on the requirements level (Pipfile) but this is in general considered as a bad practice for this use case. Having dependencies locked on pipeline unit level allows locking of a package only if it is about to be resolved. On the other hand, locking packages on requirements level creates request for the resolver to have the given requirement always present in the stack. This can have negative impact on the tested software as the tested part is not minimal and the required dependency can affect how dependency graph looks like with respect to other dependencies as well.
See Amun inspection present in thoth-station/dependency-monkey-zoo repository
which holds all the files required for scheduling and checking configuration
used for the Dependency Monkey run using the package
combinations predictor. This configuration was used to create all the version
urllib3 that can occur in a
tensorflow==2.1.0 stack (to date the inspection was created). Most notable
Pipfile holds just
tensorflow==2.1.0as a requirement for the resolver.
The pipeline configuration present in the
pipeline.confpins packages to specific version on pipeline unit level in opposite to locking on the requirements level:
steps: # Pin numpy to one version that can occur across all the stacks resolved: - configuration: only_once: false package_name: numpy name: OneVersionStride
Parameters passed to the predictor specify which package combinations should be considered during the resolution (thus predictor guides the resolution process to resolve these combinations):
package_combinations: - six - urllib3
As can be seen, using the pipeline configuration and the predictor in a specific way can lead to desired results - both can cooperate together.
The video below demonstrates the whole process in action:
The results presented in the video can be seen summarized in this following blog.