S3FileLogger

class S3FileLogger(bucket: str, experiments_folder: str, config: Optional[Dict[str, Any]] = None, gzip: bool = False)[source]

Bases: etna.loggers.file_logger.BaseFileLogger

Logger for logging files into S3 bucket.

This logger is very similar to LocalFileLogger, but works with S3 keys instead of paths at local file system.

Create instance of S3FileLogger.

Parameters

  • bucket (str) – name of the S3 bucket

  • experiments_folder (str) – path to folder to create experiment in

  • config (Optional[Dict[str, Any]]) – a dictionary-like object for saving inputs to your job, like hyperparameters for a model or settings for a data preprocessing job

  • gzip (bool) – indicator whether to use compression during saving tables or not

Raises

  • ValueError: – if environment variable endpoint_url isn’t set

  • ValueError: – if environment variable aws_access_key_id isn’t set

  • ValueError: – if environment variable aws_secret_access_key isn’t set

  • ValueError: – if bucket doesn’t exist

Inherited-members

Methods

finish_experiment(*args, **kwargs)

Finish experiment.

log(msg, **kwargs)

Log any event.

log_backtest_metrics(ts, metrics_df, ...)

Write metrics to logger.

log_backtest_run(metrics, forecast, test)

Backtest metrics from one fold to logger.

set_params(**params)

Return new object instance with modified parameters.

start_experiment([job_type, group])

Start experiment within current experiment, it is used for separate different folds during backtest.

to_dict()

Collect all information about etna object in dict.
finish_experiment(*args, **kwargs)

Finish experiment.

log(msg: Union[str, Dict[str, Any]], **kwargs)

Log any event.

This class does nothing with it, use other loggers to do it.

Parameters

  • msg (Union[str, Dict[str, Any]]) – Message or dict to log

  • kwargs – Additional parameters for particular implementation

log_backtest_metrics(ts: TSDataset, metrics_df: pandas.core.frame.DataFrame, forecast_df: pandas.core.frame.DataFrame, fold_info_df: pandas.core.frame.DataFrame)

Write metrics to logger.

Parameters

  • ts (TSDataset) – TSDataset to with backtest data

  • metrics_df (pandas.core.frame.DataFrame) – Dataframe produced with etna.pipeline.Pipeline._get_backtest_metrics()

  • forecast_df (pandas.core.frame.DataFrame) – Forecast from backtest

  • fold_info_df (pandas.core.frame.DataFrame) – Fold information from backtest

Notes

If some exception during saving is raised, then it becomes a warning.

log_backtest_run(metrics: pandas.core.frame.DataFrame, forecast: pandas.core.frame.DataFrame, test: pandas.core.frame.DataFrame)

Backtest metrics from one fold to logger.

Parameters

  • metrics (pandas.core.frame.DataFrame) – Dataframe with metrics from backtest fold

  • forecast (pandas.core.frame.DataFrame) – Dataframe with forecast

  • test (pandas.core.frame.DataFrame) – Dataframe with ground truth

Notes

If some exception during saving is raised, then it becomes a warning.

set_params(**params: dict) → etna.core.mixins.TMixin

Return new object instance with modified parameters.

Method also allows to change parameters of nested objects within the current object. For example, it is possible to change parameters of a model in a Pipeline.

Nested parameters are expected to be in a <component_1>.<...>.<parameter> form, where components are separated by a dot.

Parameters

  • **params – Estimator parameters

  • self (etna.core.mixins.TMixin) –

  • params (dict) –

Returns

New instance with changed parameters

Return type

etna.core.mixins.TMixin

Examples

>>> from etna.pipeline import Pipeline
>>> from etna.models import NaiveModel
>>> from etna.transforms import AddConstTransform
>>> model = model=NaiveModel(lag=1)
>>> transforms = [AddConstTransform(in_column="target", value=1)]
>>> pipeline = Pipeline(model, transforms=transforms, horizon=3)
>>> pipeline.set_params(**{"model.lag": 3, "transforms.0.value": 2})
Pipeline(model = NaiveModel(lag = 3, ), transforms = [AddConstTransform(in_column = 'target', value = 2, inplace = True, out_column = None, )], horizon = 3, )
start_experiment(job_type: Optional[str] = None, group: Optional[str] = None, *args, **kwargs)[source]

Start experiment within current experiment, it is used for separate different folds during backtest.

As a result, self.experiment_folder key is extended with job_type/group.

  • If job_type or group isn’t set then key is extended with one value.

  • If none of job_type and group is set then self.experiment_folder is not extended.

Parameters

  • job_type (Optional[str]) – Specify the type of run, which is useful when you’re grouping runs together into larger experiments using group.

  • group (Optional[str]) – Specify a group to organize individual runs into a larger experiment.

to_dict()

Collect all information about etna object in dict.