file_logger

Classes

BaseFileLogger()

Base logger for logging files.

LocalFileLogger(experiments_folder[, ...])

Logger for logging files into local folder.

S3FileLogger(bucket, experiments_folder[, ...])

Logger for logging files into S3 bucket.
class BaseFileLogger[source]

Base logger for logging files.

Create logger instance.

finish_experiment(*args, **kwargs)

Finish experiment.

log(msg: Union[str, Dict[str, Any]], **kwargs)[source]

Log any event.

This class does nothing with it, use other loggers to do it.

Parameters

  • msg (Union[str, Dict[str, Any]]) – Message or dict to log

  • kwargs – Additional parameters for particular implementation

log_backtest_metrics(ts: TSDataset, metrics_df: pandas.core.frame.DataFrame, forecast_df: pandas.core.frame.DataFrame, fold_info_df: pandas.core.frame.DataFrame)[source]

Write metrics to logger.

Parameters

  • ts (TSDataset) – TSDataset to with backtest data

  • metrics_df (pandas.core.frame.DataFrame) – Dataframe produced with etna.pipeline.Pipeline._get_backtest_metrics()

  • forecast_df (pandas.core.frame.DataFrame) – Forecast from backtest

  • fold_info_df (pandas.core.frame.DataFrame) – Fold information from backtest

Notes

If some exception during saving is raised, then it becomes a warning.

log_backtest_run(metrics: pandas.core.frame.DataFrame, forecast: pandas.core.frame.DataFrame, test: pandas.core.frame.DataFrame)[source]

Backtest metrics from one fold to logger.

Parameters

  • metrics (pandas.core.frame.DataFrame) – Dataframe with metrics from backtest fold

  • forecast (pandas.core.frame.DataFrame) – Dataframe with forecast

  • test (pandas.core.frame.DataFrame) – Dataframe with ground truth

Notes

If some exception during saving is raised, then it becomes a warning.

set_params(**params: dict) → etna.core.mixins.TMixin

Return new object instance with modified parameters.

Method also allows to change parameters of nested objects within the current object. For example, it is possible to change parameters of a model in a Pipeline.

Nested parameters are expected to be in a <component_1>.<...>.<parameter> form, where components are separated by a dot.

Parameters

  • **params – Estimator parameters

  • self (etna.core.mixins.TMixin) –

  • params (dict) –

Returns

New instance with changed parameters

Return type

etna.core.mixins.TMixin

Examples

>>> from etna.pipeline import Pipeline
>>> from etna.models import NaiveModel
>>> from etna.transforms import AddConstTransform
>>> model = model=NaiveModel(lag=1)
>>> transforms = [AddConstTransform(in_column="target", value=1)]
>>> pipeline = Pipeline(model, transforms=transforms, horizon=3)
>>> pipeline.set_params(**{"model.lag": 3, "transforms.0.value": 2})
Pipeline(model = NaiveModel(lag = 3, ), transforms = [AddConstTransform(in_column = 'target', value = 2, inplace = True, out_column = None, )], horizon = 3, )
abstract start_experiment(job_type: Optional[str] = None, group: Optional[str] = None, *args, **kwargs)[source]

Start experiment within current experiment, it is used for separate different folds during backtest.

Parameters

  • job_type (Optional[str]) – Specify the type of run, which is useful when you’re grouping runs together into larger experiments using group.

  • group (Optional[str]) – Specify a group to organize individual runs into a larger experiment.

to_dict()

Collect all information about etna object in dict.

class LocalFileLogger(experiments_folder: str, config: Optional[Dict[str, Any]] = None, gzip: bool = False)[source]

Logger for logging files into local folder.

It writes its result into folder like experiments_folder/2021-12-12T12-12-12, where the second part is related to datetime of starting the experiment.

After every start_experiment it creates a new subfolder job_type/group. If some of these two values are None then behaviour is little different and described in start_experiment method.

Create instance of LocalFileLogger.

Parameters

  • experiments_folder (str) – path to folder to create experiment in

  • config (Optional[Dict[str, Any]]) – a dictionary-like object for saving inputs to your job, like hyperparameters for a model or settings for a data preprocessing job

  • gzip (bool) – indicator whether to use compression during saving tables or not

finish_experiment(*args, **kwargs)

Finish experiment.

log(msg: Union[str, Dict[str, Any]], **kwargs)

Log any event.

This class does nothing with it, use other loggers to do it.

Parameters

  • msg (Union[str, Dict[str, Any]]) – Message or dict to log

  • kwargs – Additional parameters for particular implementation

log_backtest_metrics(ts: TSDataset, metrics_df: pandas.core.frame.DataFrame, forecast_df: pandas.core.frame.DataFrame, fold_info_df: pandas.core.frame.DataFrame)

Write metrics to logger.

Parameters

  • ts (TSDataset) – TSDataset to with backtest data

  • metrics_df (pandas.core.frame.DataFrame) – Dataframe produced with etna.pipeline.Pipeline._get_backtest_metrics()

  • forecast_df (pandas.core.frame.DataFrame) – Forecast from backtest

  • fold_info_df (pandas.core.frame.DataFrame) – Fold information from backtest

Notes

If some exception during saving is raised, then it becomes a warning.

log_backtest_run(metrics: pandas.core.frame.DataFrame, forecast: pandas.core.frame.DataFrame, test: pandas.core.frame.DataFrame)

Backtest metrics from one fold to logger.

Parameters

  • metrics (pandas.core.frame.DataFrame) – Dataframe with metrics from backtest fold

  • forecast (pandas.core.frame.DataFrame) – Dataframe with forecast

  • test (pandas.core.frame.DataFrame) – Dataframe with ground truth

Notes

If some exception during saving is raised, then it becomes a warning.

set_params(**params: dict) → etna.core.mixins.TMixin

Return new object instance with modified parameters.

Method also allows to change parameters of nested objects within the current object. For example, it is possible to change parameters of a model in a Pipeline.

Nested parameters are expected to be in a <component_1>.<...>.<parameter> form, where components are separated by a dot.

Parameters

  • **params – Estimator parameters

  • self (etna.core.mixins.TMixin) –

  • params (dict) –

Returns

New instance with changed parameters

Return type

etna.core.mixins.TMixin

Examples

>>> from etna.pipeline import Pipeline
>>> from etna.models import NaiveModel
>>> from etna.transforms import AddConstTransform
>>> model = model=NaiveModel(lag=1)
>>> transforms = [AddConstTransform(in_column="target", value=1)]
>>> pipeline = Pipeline(model, transforms=transforms, horizon=3)
>>> pipeline.set_params(**{"model.lag": 3, "transforms.0.value": 2})
Pipeline(model = NaiveModel(lag = 3, ), transforms = [AddConstTransform(in_column = 'target', value = 2, inplace = True, out_column = None, )], horizon = 3, )
start_experiment(job_type: Optional[str] = None, group: Optional[str] = None, *args, **kwargs)[source]

Start experiment within current experiment, it is used for separate different folds during backtest.

As a result, within self.experiment_folder subfolder job_type/group is created.

  • If job_type or group isn’t set then only one-level subfolder is created.

  • If none of job_type and group is set then experiment logs files into self.experiment_folder.

Parameters

  • job_type (Optional[str]) – Specify the type of run, which is useful when you’re grouping runs together into larger experiments using group.

  • group (Optional[str]) – Specify a group to organize individual runs into a larger experiment.

to_dict()

Collect all information about etna object in dict.

class S3FileLogger(bucket: str, experiments_folder: str, config: Optional[Dict[str, Any]] = None, gzip: bool = False)[source]

Logger for logging files into S3 bucket.

This logger is very similar to LocalFileLogger, but works with S3 keys instead of paths at local file system.

Create instance of S3FileLogger.

Parameters

  • bucket (str) – name of the S3 bucket

  • experiments_folder (str) – path to folder to create experiment in

  • config (Optional[Dict[str, Any]]) – a dictionary-like object for saving inputs to your job, like hyperparameters for a model or settings for a data preprocessing job

  • gzip (bool) – indicator whether to use compression during saving tables or not

Raises

  • ValueError: – if environment variable endpoint_url isn’t set

  • ValueError: – if environment variable aws_access_key_id isn’t set

  • ValueError: – if environment variable aws_secret_access_key isn’t set

  • ValueError: – if bucket doesn’t exist

finish_experiment(*args, **kwargs)

Finish experiment.

log(msg: Union[str, Dict[str, Any]], **kwargs)

Log any event.

This class does nothing with it, use other loggers to do it.

Parameters

  • msg (Union[str, Dict[str, Any]]) – Message or dict to log

  • kwargs – Additional parameters for particular implementation

log_backtest_metrics(ts: TSDataset, metrics_df: pandas.core.frame.DataFrame, forecast_df: pandas.core.frame.DataFrame, fold_info_df: pandas.core.frame.DataFrame)

Write metrics to logger.

Parameters

  • ts (TSDataset) – TSDataset to with backtest data

  • metrics_df (pandas.core.frame.DataFrame) – Dataframe produced with etna.pipeline.Pipeline._get_backtest_metrics()

  • forecast_df (pandas.core.frame.DataFrame) – Forecast from backtest

  • fold_info_df (pandas.core.frame.DataFrame) – Fold information from backtest

Notes

If some exception during saving is raised, then it becomes a warning.

log_backtest_run(metrics: pandas.core.frame.DataFrame, forecast: pandas.core.frame.DataFrame, test: pandas.core.frame.DataFrame)

Backtest metrics from one fold to logger.

Parameters

  • metrics (pandas.core.frame.DataFrame) – Dataframe with metrics from backtest fold

  • forecast (pandas.core.frame.DataFrame) – Dataframe with forecast

  • test (pandas.core.frame.DataFrame) – Dataframe with ground truth

Notes

If some exception during saving is raised, then it becomes a warning.

set_params(**params: dict) → etna.core.mixins.TMixin

Return new object instance with modified parameters.

Method also allows to change parameters of nested objects within the current object. For example, it is possible to change parameters of a model in a Pipeline.

Nested parameters are expected to be in a <component_1>.<...>.<parameter> form, where components are separated by a dot.

Parameters

  • **params – Estimator parameters

  • self (etna.core.mixins.TMixin) –

  • params (dict) –

Returns

New instance with changed parameters

Return type

etna.core.mixins.TMixin

Examples

>>> from etna.pipeline import Pipeline
>>> from etna.models import NaiveModel
>>> from etna.transforms import AddConstTransform
>>> model = model=NaiveModel(lag=1)
>>> transforms = [AddConstTransform(in_column="target", value=1)]
>>> pipeline = Pipeline(model, transforms=transforms, horizon=3)
>>> pipeline.set_params(**{"model.lag": 3, "transforms.0.value": 2})
Pipeline(model = NaiveModel(lag = 3, ), transforms = [AddConstTransform(in_column = 'target', value = 2, inplace = True, out_column = None, )], horizon = 3, )
start_experiment(job_type: Optional[str] = None, group: Optional[str] = None, *args, **kwargs)[source]

Start experiment within current experiment, it is used for separate different folds during backtest.

As a result, self.experiment_folder key is extended with job_type/group.

  • If job_type or group isn’t set then key is extended with one value.

  • If none of job_type and group is set then self.experiment_folder is not extended.

Parameters

  • job_type (Optional[str]) – Specify the type of run, which is useful when you’re grouping runs together into larger experiments using group.

  • group (Optional[str]) – Specify a group to organize individual runs into a larger experiment.

to_dict()

Collect all information about etna object in dict.