thoth.adviser.steps package

Subpackages

Submodules

thoth.adviser.steps.cve module

Penalize stacks with a CVE.

class thoth.adviser.steps.cve.CvePenalizationStep(*, unit_run: bool = False, configuration: Dict[str, Any] = NOTHING)[source]

Bases: thoth.adviser.step.Step

Penalization based on CVE being present in stack.

CONFIGURATION_DEFAULT = {'cve_penalization': -0.2, 'multi_package_resolution': False, 'package_name': None}
CONFIGURATION_SCHEMA = <Schema({'package_name': None, 'cve_penalization': <class 'float'>, 'multi_package_resolution': False}, extra=PREVENT_EXTRA, required=False) object>
pre_run() → None[source]

Initialize this pipeline unit before running.

run(_: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[float, List[Dict[str, str]]]][source]

Penalize stacks with a CVE.

classmethod should_include(builder_context: PipelineBuilderContext) → Generator[Dict[str, Any], None, None][source]

Remove CVEs only for advised stacks.

thoth.adviser.steps.dropout module

Perform a drop out of a new state expansion, randomly.

class thoth.adviser.steps.dropout.DropoutStep(*, unit_run: bool = False, configuration: Dict[str, Any] = NOTHING)[source]

Bases: thoth.adviser.step.Step

A step that drops a state transition with a certain probability.

CONFIGURATION_DEFAULT = {'multi_package_resolution': False, 'package_name': None, 'probability': 0.9}
CONFIGURATION_SCHEMA = <Schema({'package_name': None, 'cve_penalization': <class 'float'>, 'multi_package_resolution': False}, extra=PREVENT_EXTRA, required=False) object>
run(state: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Do not accept new state, randomly.

classmethod should_include(builder_context: PipelineBuilderContext) → Generator[Dict[str, Any], None, None][source]

Do not register the dropout step.

thoth.adviser.steps.security_indicators module

Score based on security indicators aggregated.

class thoth.adviser.steps.security_indicators.SecurityIndicatorStep(*, unit_run: bool = False, configuration: Dict[str, Any] = NOTHING)[source]

Bases: thoth.adviser.step.Step

A step that scores a state based on security info aggregated.

CONFIGURATION_DEFAULT = {'function_scaling': 0.001, 'high_confidence_weight': 1.0, 'high_severity_weight': 100.0, 'low_confidence_weight': 0.1, 'low_severity_weight': 1.0, 'medium_confidence_weight': 0.5, 'medium_severity_weight': 10.0, 'multi_package_resolution': False, 'package_name': None, 'si_score_weight': 0.5}
CONFIGURATION_SCHEMA = <Schema({'high_confidence_weight': <class 'float'>, 'high_severity_weight': <class 'float'>, 'low_confidence_weight': <class 'float'>, 'low_severity_weight': <class 'float'>, 'medium_confidence_weight': <class 'float'>, 'medium_severity_weight': <class 'float'>, 'multi_package_resolution': <class 'bool'>, 'package_name': None, 'si_score_weight': <class 'float'>, 'function_scaling': <class 'float'>}, extra=PREVENT_EXTRA, required=False) object>
pre_run() → None[source]

Initialize this pipeline step before running the pipeline.

run(state: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Score package based on security indicators gathered, do not include if not analyzed.

classmethod should_include(builder_context: PipelineBuilderContext) → Generator[Dict[str, Any], None, None][source]

Register only if we are explicitly recommending secure stacks.

Module contents

Implementation of steps used during resolution.

class thoth.adviser.steps.CvePenalizationStep(*, unit_run: bool = False, configuration: Dict[str, Any] = NOTHING)[source]

Bases: thoth.adviser.step.Step

Penalization based on CVE being present in stack.

CONFIGURATION_DEFAULT = {'cve_penalization': -0.2, 'multi_package_resolution': False, 'package_name': None}
CONFIGURATION_SCHEMA = <Schema({'package_name': None, 'cve_penalization': <class 'float'>, 'multi_package_resolution': False}, extra=PREVENT_EXTRA, required=False) object>
pre_run() → None[source]

Initialize this pipeline unit before running.

run(_: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[float, List[Dict[str, str]]]][source]

Penalize stacks with a CVE.

classmethod should_include(builder_context: PipelineBuilderContext) → Generator[Dict[str, Any], None, None][source]

Remove CVEs only for advised stacks.

class thoth.adviser.steps.DropoutStep(*, unit_run: bool = False, configuration: Dict[str, Any] = NOTHING)[source]

Bases: thoth.adviser.step.Step

A step that drops a state transition with a certain probability.

CONFIGURATION_DEFAULT = {'multi_package_resolution': False, 'package_name': None, 'probability': 0.9}
CONFIGURATION_SCHEMA = <Schema({'package_name': None, 'cve_penalization': <class 'float'>, 'multi_package_resolution': False}, extra=PREVENT_EXTRA, required=False) object>
run(state: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Do not accept new state, randomly.

classmethod should_include(builder_context: PipelineBuilderContext) → Generator[Dict[str, Any], None, None][source]

Do not register the dropout step.

class thoth.adviser.steps.SecurityIndicatorStep(*, unit_run: bool = False, configuration: Dict[str, Any] = NOTHING)[source]

Bases: thoth.adviser.step.Step

A step that scores a state based on security info aggregated.

CONFIGURATION_DEFAULT = {'function_scaling': 0.001, 'high_confidence_weight': 1.0, 'high_severity_weight': 100.0, 'low_confidence_weight': 0.1, 'low_severity_weight': 1.0, 'medium_confidence_weight': 0.5, 'medium_severity_weight': 10.0, 'multi_package_resolution': False, 'package_name': None, 'si_score_weight': 0.5}
CONFIGURATION_SCHEMA = <Schema({'high_confidence_weight': <class 'float'>, 'high_severity_weight': <class 'float'>, 'low_confidence_weight': <class 'float'>, 'low_severity_weight': <class 'float'>, 'medium_confidence_weight': <class 'float'>, 'medium_severity_weight': <class 'float'>, 'multi_package_resolution': <class 'bool'>, 'package_name': None, 'si_score_weight': <class 'float'>, 'function_scaling': <class 'float'>}, extra=PREVENT_EXTRA, required=False) object>
pre_run() → None[source]

Initialize this pipeline step before running the pipeline.

run(state: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Score package based on security indicators gathered, do not include if not analyzed.

classmethod should_include(builder_context: PipelineBuilderContext) → Generator[Dict[str, Any], None, None][source]

Register only if we are explicitly recommending secure stacks.

class thoth.adviser.steps.MockScoreStep(*, unit_run: bool = False, configuration: Dict[str, Any] = NOTHING)[source]

Bases: thoth.adviser.step.Step

A step that is mocking scoring of packages.

CONFIGURATION_DEFAULT = {'assign_probability': 0.75, 'multi_package_resolution': False, 'package_name': None}
CONFIGURATION_SCHEMA = <Schema({'package_name': Any(<class 'str'>, None, msg=None), 'assign_probability': <class 'float'>, 'multi_package_resolution': <class 'bool'>}, extra=PREVENT_EXTRA, required=False) object>
post_run() → None[source]

Print the generated history after the run.

pre_run() → None[source]

Initialize self, before each run.

run(_: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Score the given package regardless of the state.

classmethod should_include(builder_context: PipelineBuilderContext) → Generator[Dict[str, Any], None, None][source]

Register self, never.

class thoth.adviser.steps.SetScoreStep(*, unit_run: bool = False, configuration: Dict[str, Any] = NOTHING)[source]

Bases: thoth.adviser.step.Step

A step that is setting score for packages.

CONFIGURATION_DEFAULT = {'index_url': None, 'multi_package_resolution': False, 'package_name': None, 'package_version': None, 'score': None}
CONFIGURATION_SCHEMA = <Schema({'multi_package_resolution': <class 'bool'>, 'package_name': <class 'str'>, 'index_url': Any(<class 'str'>, None, msg=None), 'package_version': Any(<class 'str'>, None, msg=None), 'score': Any(<class 'float'>, None, msg=None)}, extra=PREVENT_EXTRA, required=False) object>
pre_run() → None[source]

Initialize this pipeline unit before each run.

run(_: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Score the given package.

classmethod should_include(builder_context: PipelineBuilderContext) → Generator[Dict[str, Any], None, None][source]

Register self, never.

class thoth.adviser.steps.GenerateScoreStep(*, unit_run: bool = False, configuration: Dict[str, Any] = NOTHING)[source]

Bases: thoth.adviser.step.Step

A step that is assigning scores in a deterministic way.

This unit can be used to measure assigning score in a deterministic way across multiple runs without a need to store all the score for packages.

CONFIGURATION_DEFAULT = {'assign_probability': 0.75, 'buffer_size': 1024, 'multi_package_resolution': False, 'package_name': None, 'seed': 42}
CONFIGURATION_SCHEMA = <Schema({'assign_probability': <class 'float'>, 'buffer_size': <class 'int'>, 'package_name': Any(<class 'str'>, None, msg=None), 'seed': <class 'int'>, 'multi_package_resolution': <class 'bool'>}, extra=PREVENT_EXTRA, required=False) object>
post_run() → None[source]

Print the generated scores on finish to stdout.

pre_run() → None[source]

Initialize this pipeline units before each run.

run(_: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Score the given package.

classmethod should_include(builder_context: PipelineBuilderContext) → Generator[Dict[str, Any], None, None][source]

Register self, never.

class thoth.adviser.steps.TensorFlowRemoveSciPyStep(*, unit_run: bool = False, configuration: Dict[str, Any] = NOTHING)[source]

Bases: thoth.adviser.step.Step

A step that removes SciPy dependency from a TensorFlow>2.1<=2.3 stack.

https://github.com/tensorflow/tensorflow/pull/41866 https://github.com/tensorflow/tensorflow/issues/40884 https://github.com/tensorflow/tensorflow/issues/35709 https://github.com/tensorflow/tensorflow/issues/41941 https://github.com/tensorflow/tensorflow/pull/40789

CONFIGURATION_DEFAULT = {'multi_package_resolution': False, 'package_name': 'scipy'}
pre_run() → None[source]

Initialize this pipeline unit before each run.

run(state: thoth.adviser.state.State, package_version: thoth.python.package_version.PackageVersion) → Optional[Tuple[Optional[float], Optional[List[Dict[str, str]]]]][source]

Remove SciPy dependency from a TensorFlow>2.1<=2.3 stack.

classmethod should_include(builder_context: PipelineBuilderContext) → Generator[Dict[Any, Any], None, None][source]

Include this unit in adviser, except for latest recommendations.