thoth.adviser.steps package

Submodules

thoth.adviser.steps.aicoe module

A step to prioritize releases from AICoE.

class thoth.adviser.steps.aicoe.AICoEReleasesStep(*, configuration: Dict[str, Any] = NOTHING, name: str = None, unit_run: bool = False)[source]

Bases: thoth.adviser.step.Step

Prioritize releases from AICoE.

CONFIGURATION_DEFAULT = {'multi_package_resolution': False, 'package_name': None}
run(_: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[float, List[Dict[str, str]]]][source]

Prioritize releases from AICoE.

classmethod should_include(builder_context: PipelineBuilderContext) → Optional[Dict[str, Any]][source]

Register self, for non-latest adviser runs.

thoth.adviser.steps.cve module

Penalize stacks with a CVE.

class thoth.adviser.steps.cve.CvePenalizationStep(*, configuration: Dict[str, Any] = NOTHING, name: str = None, unit_run: bool = False)[source]

Bases: thoth.adviser.step.Step

Penalization based on CVE being present in stack.

CONFIGURATION_DEFAULT = {'cve_penalization': -0.2, 'multi_package_resolution': False, 'package_name': None}
CONFIGURATION_SCHEMA = <Schema({'package_name': None, 'cve_penalization': <class 'float'>, 'multi_package_resolution': False}, extra=PREVENT_EXTRA, required=False) object>
pre_run() → None[source]

Initialize this pipeline unit before running.

run(_: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[float, List[Dict[str, str]]]][source]

Penalize stacks with a CVE.

classmethod should_include(builder_context: PipelineBuilderContext) → Optional[Dict[str, Any]][source]

Remove CVEs only for advised stacks.

thoth.adviser.steps.dropout module

Perform a drop out of a new state expansion, randomly.

class thoth.adviser.steps.dropout.DropoutStep(*, configuration: Dict[str, Any] = NOTHING, name: str = None, unit_run: bool = False)[source]

Bases: thoth.adviser.step.Step

A step that drops a state transition with a certain probability.

CONFIGURATION_DEFAULT = {'multi_package_resolution': False, 'package_name': None, 'probability': 0.9}
CONFIGURATION_SCHEMA = <Schema({'package_name': None, 'cve_penalization': <class 'float'>, 'multi_package_resolution': False}, extra=PREVENT_EXTRA, required=False) object>
run(state: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Do not accept new state, randomly.

classmethod should_include(builder_context: PipelineBuilderContext) → Optional[Dict[str, Any]][source]

Do not register the dropout step.

thoth.adviser.steps.security_indicators module

Score based on security indicators aggregated.

class thoth.adviser.steps.security_indicators.SecurityIndicatorStep(*, configuration: Dict[str, Any] = NOTHING, name: str = None, unit_run: bool = False)[source]

Bases: thoth.adviser.step.Step

A step that scores a state based on security info aggregated.

CONFIGURATION_DEFAULT = {'high_confidence_weight': 1.0, 'high_severity_weight': 100.0, 'low_confidence_weight': 0.1, 'low_severity_weight': 1.0, 'medium_confidence_weight': 0.5, 'medium_severity_weight': 10.0, 'multi_package_resolution': False, 'package_name': None, 'si_reward_weight': 0.5}
CONFIGURATION_SCHEMA = <Schema({'high_confidence_weight': <class 'float'>, 'high_severity_weight': <class 'float'>, 'low_confidence_weight': <class 'float'>, 'low_severity_weight': <class 'float'>, 'medium_confidence_weight': <class 'float'>, 'medium_severity_weight': <class 'float'>, 'multi_package_resolution': <class 'bool'>, 'package_name': None, 'si_reward_weight': <class 'float'>}, extra=PREVENT_EXTRA, required=False) object>
pre_run() → None[source]

Initialize this pipeline step before running the pipeline.

run(state: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Score package based on security indicators gathered, do not include if not analyzed.

classmethod should_include(builder_context: PipelineBuilderContext) → Optional[Dict[str, Any]][source]

Register only if we are explicitly recommending secure stacks.

Module contents

Implementation of steps used during resolution.

class thoth.adviser.steps.AICoEReleasesStep(*, configuration: Dict[str, Any] = NOTHING, name: str = None, unit_run: bool = False)[source]

Bases: thoth.adviser.step.Step

Prioritize releases from AICoE.

CONFIGURATION_DEFAULT = {'multi_package_resolution': False, 'package_name': None}
run(_: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[float, List[Dict[str, str]]]][source]

Prioritize releases from AICoE.

classmethod should_include(builder_context: PipelineBuilderContext) → Optional[Dict[str, Any]][source]

Register self, for non-latest adviser runs.

class thoth.adviser.steps.CvePenalizationStep(*, configuration: Dict[str, Any] = NOTHING, name: str = None, unit_run: bool = False)[source]

Bases: thoth.adviser.step.Step

Penalization based on CVE being present in stack.

CONFIGURATION_DEFAULT = {'cve_penalization': -0.2, 'multi_package_resolution': False, 'package_name': None}
CONFIGURATION_SCHEMA = <Schema({'package_name': None, 'cve_penalization': <class 'float'>, 'multi_package_resolution': False}, extra=PREVENT_EXTRA, required=False) object>
pre_run() → None[source]

Initialize this pipeline unit before running.

run(_: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[float, List[Dict[str, str]]]][source]

Penalize stacks with a CVE.

classmethod should_include(builder_context: PipelineBuilderContext) → Optional[Dict[str, Any]][source]

Remove CVEs only for advised stacks.

class thoth.adviser.steps.DropoutStep(*, configuration: Dict[str, Any] = NOTHING, name: str = None, unit_run: bool = False)[source]

Bases: thoth.adviser.step.Step

A step that drops a state transition with a certain probability.

CONFIGURATION_DEFAULT = {'multi_package_resolution': False, 'package_name': None, 'probability': 0.9}
CONFIGURATION_SCHEMA = <Schema({'package_name': None, 'cve_penalization': <class 'float'>, 'multi_package_resolution': False}, extra=PREVENT_EXTRA, required=False) object>
run(state: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Do not accept new state, randomly.

classmethod should_include(builder_context: PipelineBuilderContext) → Optional[Dict[str, Any]][source]

Do not register the dropout step.

class thoth.adviser.steps.SecurityIndicatorStep(*, configuration: Dict[str, Any] = NOTHING, name: str = None, unit_run: bool = False)[source]

Bases: thoth.adviser.step.Step

A step that scores a state based on security info aggregated.

CONFIGURATION_DEFAULT = {'high_confidence_weight': 1.0, 'high_severity_weight': 100.0, 'low_confidence_weight': 0.1, 'low_severity_weight': 1.0, 'medium_confidence_weight': 0.5, 'medium_severity_weight': 10.0, 'multi_package_resolution': False, 'package_name': None, 'si_reward_weight': 0.5}
CONFIGURATION_SCHEMA = <Schema({'high_confidence_weight': <class 'float'>, 'high_severity_weight': <class 'float'>, 'low_confidence_weight': <class 'float'>, 'low_severity_weight': <class 'float'>, 'medium_confidence_weight': <class 'float'>, 'medium_severity_weight': <class 'float'>, 'multi_package_resolution': <class 'bool'>, 'package_name': None, 'si_reward_weight': <class 'float'>}, extra=PREVENT_EXTRA, required=False) object>
pre_run() → None[source]

Initialize this pipeline step before running the pipeline.

run(state: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Score package based on security indicators gathered, do not include if not analyzed.

classmethod should_include(builder_context: PipelineBuilderContext) → Optional[Dict[str, Any]][source]

Register only if we are explicitly recommending secure stacks.

class thoth.adviser.steps.MockScoreStep(*, configuration: Dict[str, Any] = NOTHING, name: str = None, unit_run: bool = False)[source]

Bases: thoth.adviser.step.Step

A step that is mocking scoring of packages.

CONFIGURATION_DEFAULT = {'assign_probability': 0.75, 'multi_package_resolution': False, 'package_name': None}
CONFIGURATION_SCHEMA = <Schema({'package_name': Any(<class 'str'>, None, msg=None), 'assign_probability': <class 'float'>, 'multi_package_resolution': <class 'bool'>}, extra=PREVENT_EXTRA, required=False) object>
post_run() → None[source]

Print the generated history after the run.

pre_run() → None[source]

Initialize self, before each run.

run(_: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Score the given package regardless of the state.

classmethod should_include(builder_context: PipelineBuilderContext) → Optional[Dict[str, Any]][source]

Register self, never.

class thoth.adviser.steps.SetScoreStep(*, configuration: Dict[str, Any] = NOTHING, name: str = None, unit_run: bool = False)[source]

Bases: thoth.adviser.step.Step

A step that is setting score for packages.

CONFIGURATION_DEFAULT = {'index_url': None, 'multi_package_resolution': False, 'package_name': None, 'package_version': None, 'score': None}
CONFIGURATION_SCHEMA = <Schema({'multi_package_resolution': <class 'bool'>, 'package_name': <class 'str'>, 'index_url': Any(<class 'str'>, None, msg=None), 'package_version': Any(<class 'str'>, None, msg=None), 'score': Any(<class 'float'>, None, msg=None)}, extra=PREVENT_EXTRA, required=False) object>
pre_run() → None[source]

Initialize this pipeline unit before each run.

run(_: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Score the given package.

classmethod should_include(builder_context: PipelineBuilderContext) → Optional[Dict[str, Any]][source]

Register self, never.

class thoth.adviser.steps.GenerateScoreStep(*, configuration: Dict[str, Any] = NOTHING, name: str = None, unit_run: bool = False)[source]

Bases: thoth.adviser.step.Step

A step that is assigning scores in a deterministic way.

This unit can be used to measure assigning score in a deterministic way across multiple runs without a need to store all the score for packages.

CONFIGURATION_DEFAULT = {'assign_probability': 0.75, 'buffer_size': 1024, 'multi_package_resolution': False, 'package_name': None, 'seed': 42}
CONFIGURATION_SCHEMA = <Schema({'assign_probability': <class 'float'>, 'buffer_size': <class 'int'>, 'package_name': Any(<class 'str'>, None, msg=None), 'seed': <class 'int'>, 'multi_package_resolution': <class 'bool'>}, extra=PREVENT_EXTRA, required=False) object>
post_run() → None[source]

Print the generated scores on finish to stdout.

pre_run() → None[source]

Initialize this pipeline units before each run.

run(_: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Score the given package.

classmethod should_include(builder_context: PipelineBuilderContext) → Optional[Dict[str, Any]][source]

Register self, never.

class thoth.adviser.steps.TensorFlow21Urllib3Step(*, configuration: Dict[str, Any] = NOTHING, name: str = None, unit_run: bool = False)[source]

Bases: thoth.adviser.step.Step

A step that suggests not to use TensorFlow 2.1 with specific urllib3 versions that cause six import issues.

CONFIGURATION_DEFAULT = {'multi_package_resolution': False, 'package_name': 'urllib3'}
pre_run() → None[source]

Initialize this pipeline unit before each run.

run(state: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Suggest not to use TensorFlow 2.1 with specific urllib3 as issues with six were spotted on imports.

classmethod should_include(builder_context: PipelineBuilderContext) → Optional[Dict[str, Any]][source]

Register this pipeline unit for adviser if not using latest recommendations.

class thoth.adviser.steps.TensorFlow21H5pyStep(*, configuration: Dict[str, Any] = NOTHING, name: str = None, unit_run: bool = False)[source]

Bases: thoth.adviser.step.Step

A step that suggests not to use TensorFlow 2.1/2.3 with specific h5py versions that cause issues.

See overpinning issue reported:

CONFIGURATION_DEFAULT = {'multi_package_resolution': False, 'package_name': 'h5py'}
pre_run() → None[source]

Initialize this pipeline unit before each run.

run(state: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Suggest not to use TensorFlow 2.1/2.3 with h5py>=3.

classmethod should_include(builder_context: PipelineBuilderContext) → Optional[Dict[str, Any]][source]

Register this pipeline unit for adviser if not using latest recommendations.

class thoth.adviser.steps.TensorFlow22ProbabilityStep(*, configuration: Dict[str, Any] = NOTHING, name: str = None, unit_run: bool = False)[source]

Bases: thoth.adviser.step.Step

Suggest not to use TensorFlow 2.2 with tensorflow-probability.

https://github.com/tensorflow/tensorflow/issues/40584

CONFIGURATION_DEFAULT = {'multi_package_resolution': False, 'package_name': 'tensorflow-probability'}
pre_run() → None[source]

Initialize this pipeline unit before each run.

run(state: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Suggest not to use TensorFlow 2.2 with with tensorflow-probability.

classmethod should_include(builder_context: PipelineBuilderContext) → Optional[Dict[str, Any]][source]

Register this pipeline unit for adviser if not using latest recommendations.

class thoth.adviser.steps.TensorFlowAVX2Step(*, configuration: Dict[str, Any] = NOTHING, name: str = None, unit_run: bool = False)[source]

Bases: thoth.adviser.step.Step

A step that recommends AICoE TensorFlow builds optimized for AVX2 enabled CPU processors.

AVX2_CPUS = frozenset({(6, 5), (6, 6), (6, 10), (6, 12), (6, 13), (6, 14), (6, 15)})
CONFIGURATION_DEFAULT = {'multi_package_resolution': False, 'package_name': 'tensorflow'}
run(state: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Recommend TensorFlow builds optimized for AVX2 enabled CPU processors.

classmethod should_include(builder_context: PipelineBuilderContext) → Optional[Dict[str, Any]][source]

Register this pipeline unit for adviser and stable/testing recommendation types.

class thoth.adviser.steps.TensorFlow113NumPyStep(*, configuration: Dict[str, Any] = NOTHING, name: str = None, unit_run: bool = False)[source]

Bases: thoth.adviser.step.Step

Suggest to use TensorFlow 1.13.1 with NumPy>=1.16.0.

https://github.com/tensorflow/tensorflow/issues/25636

CONFIGURATION_DEFAULT = {'multi_package_resolution': False, 'package_name': 'numpy'}
pre_run() → None[source]

Initialize this pipeline unit before each run.

run(state: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Suggest to use TensorFlow 1.13.1 with NumPy>=1.16.0.

classmethod should_include(builder_context: PipelineBuilderContext) → Optional[Dict[str, Any]][source]

Register this pipeline unit for adviser if not using latest recommendations.

class thoth.adviser.steps.TensorFlow114GastStep(*, configuration: Dict[str, Any] = NOTHING, name: str = None, unit_run: bool = False)[source]

Bases: thoth.adviser.step.Step

Suggest not to use gast>0.2.2 with TensorFlow<=1.14.

https://github.com/tensorflow/tensorflow/pull/32989 https://github.com/tensorflow/tensorflow/issues/32319

CONFIGURATION_DEFAULT = {'multi_package_resolution': False, 'package_name': 'gast'}
pre_run() → None[source]

Initialize this pipeline unit before each run.

run(state: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Suggest not to use gast>0.2.2 with TensorFlow<1.14.

classmethod should_include(builder_context: PipelineBuilderContext) → Optional[Dict[str, Any]][source]

Register this pipeline unit for adviser if not using latest recommendations.

class thoth.adviser.steps.TensorFlow22NumPyStep(*, configuration: Dict[str, Any] = NOTHING, name: str = None, unit_run: bool = False)[source]

Bases: thoth.adviser.step.Step

Suggest to use TensorFlow <2.3>=2.0 with NumPy<1.19.0.

https://github.com/numpy/numpy/pull/15355 https://github.com/tensorflow/tensorflow/commit/79518facb4b857af9d9d5df2da463fdbf7eb0e3e https://github.com/tensorflow/tensorflow/issues/41902

CONFIGURATION_DEFAULT = {'multi_package_resolution': False, 'package_name': 'numpy'}
pre_run() → None[source]

Initialize this pipeline unit before each run.

run(state: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Suggest to use TensorFlow <2.3>=2.0 with NumPy<1.19.0.

classmethod should_include(builder_context: PipelineBuilderContext) → Optional[Dict[str, Any]][source]

Register this pipeline unit for adviser if not using latest recommendations.

class thoth.adviser.steps.TensorFlowRemoveSciPyStep(*, configuration: Dict[str, Any] = NOTHING, name: str = None, unit_run: bool = False)[source]

Bases: thoth.adviser.step.Step

A step that removes SciPy dependency from a TensorFlow>2.1<=2.3 stack.

https://github.com/tensorflow/tensorflow/pull/41866 https://github.com/tensorflow/tensorflow/issues/40884 https://github.com/tensorflow/tensorflow/issues/35709 https://github.com/tensorflow/tensorflow/issues/41941 https://github.com/tensorflow/tensorflow/pull/40789

CONFIGURATION_DEFAULT = {'multi_package_resolution': False, 'package_name': 'scipy'}
pre_run() → None[source]

Initialize this pipeline unit before each run.

run(state: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Remove SciPy dependency from a TensorFlow>2.1<=2.3 stack.

classmethod should_include(builder_context: PipelineBuilderContext) → Optional[Dict[Any, Any]][source]

Include this unit in adviser, except for latest recommendations.