# archai.common package

## Submodules

## archai.common.apex_utils module

class `archai.common.apex_utils.``ApexUtils`(apex_config: archai.common.config.Config, logger: Optional[archai.common.ordereddict_logger.OrderedDictLogger])

  Bases: `object`

  `backward`(loss: torch.Tensor, multi_optim: archai.common.multi_optim.MultiOptim) → None
  `barrier`() → None
  `clip_grad`(clip: float, model: torch.nn.modules.module.Module, multi_optim: archai.common.multi_optim.MultiOptim) → None
  `is_dist`() → bool
  `is_master`() → bool
  `is_mixed`() → bool
  `is_ray`() → bool
  `load_state_dict`(state_dict)
  `reduce`(val, op='mean')
  `state_dict`()
  `sync_devices`() → None
  `to_amp`(model: torch.nn.modules.module.Module, multi_optim: archai.common.multi_optim.MultiOptim, batch_size: int) → torch.nn.modules.module.Module

## archai.common.atomic_file_handler module

class `archai.common.atomic_file_handler.``AtomicFileHandler`(filename, encoding=None, save_delay=30.0)

  Bases: `logging.Handler`

  This class opens and writes entire file instead of appending one line at a time

  `close`()

    Tidy up any resources used by the handler.

    This version removes the handler from an internal map of handlers, _handlers, which is used for handler lookup by name. Subclasses should ensure that this gets called from overridden close() methods.

  `emit`(record)

    Do whatever it takes to actually log the specified logging record.

    This version is intended to be implemented by subclasses and so raises a NotImplementedError.

  `flush`()

    Ensure all logging output has been flushed.

    This version does nothing and is intended to be implemented by subclasses.

  `terminator` = '\n'

## archai.common.checkpoint module

class `archai.common.checkpoint.``CheckPoint`(conf_checkpoint: archai.common.config.Config, load_existing: bool)

  Bases: `collections.UserDict`

  Callback based checkpoint model.

  Start new checkpoint by calling new() and save it by calling commit(). This class is also dictionary. Items that needs be saved can be done so by setting key, value pairs after new(). As any dictionary key is set, checkpoint becomes dirty. On commit(), dictionary is saved and emptied. Invariant: checkpoint remains dirty until commit() is called.

  `commit`() → None
  `is_empty`() → bool
  `load_existing`() → bool
  `new`(*kargs, **kvargs) → None
  `subscribe`(callback: Callable) → None

## archai.common.cocob module

The code below is directly from https://raw.githubusercontent.com/IssamLaradji/sls/master/others/cocob.py Two coin betting optimization algorithms are implemented here : Cocob Backprop: https://arxiv.org/pdf/1705.07795.pdf Cocob through Ons: https://arxiv.org/pdf/1705.07795.pdf both of which do not require any learning rates and yet have optimal convergence gauarantees for non-smooth convex functions.

Cocob-Ons is an experimental variation from paper. Please don’t use it yet.

Please check http://francesco.orabona.com/papers/slides_cocob.pdf for simple explanation for going from coin betting game to convex optimization. Both algorithms are similar except the coin betting strategy used.

class `archai.common.cocob.``CocobBackprop`(params, alpha=100.0, eps=1e-08)

  Bases: `torch.optim.optimizer.Optimizer`

  Implements Cocob-Backprop .

  It has been proposed in Training Deep Networks without Learning Rates Through Coin Betting.

  Arguments:
    params (iterable): iterable of parameters to optimize or dicts defining

      parameter groups

    alpha (float, optional): positive number to adjust betting fraction.

      Theoretical convergence gauarantee does not depend on choice of alpha (default: 100.0)

  `step`(closure=None)

    Performs a single optimization step (parameter update).

    Arguments:
      closure (callable): A closure that reevaluates the model and

        returns the loss. Optional for most optimizers.

class `archai.common.cocob.``CocobOns`(params, eps=1e-08)

  Bases: `torch.optim.optimizer.Optimizer`

  Implements Coin-Betting through ONS .

  It has been proposed in Black-Box Reductions for Parameter-free Online Learning in Banach Spaces.

  Cocob-Ons is an experimental variation from the paper. Do not use it yet.

  Arguments:
    params (iterable): iterable of parameters to optimize or dicts defining

      parameter groups

    eps (float, optional): positive initial wealth for betting algorithm.

      Theoretical convergence gauarantee does not depend on choice of eps (default: 1e-8)

  `step`(closure=None)

    Performs a single optimization step (parameter update).

    Arguments:
      closure (callable): A closure that reevaluates the model and

        returns the loss. Optional for most optimizers.

## archai.common.common module

class `archai.common.common.``CommonState`

  Bases: `object`

class `archai.common.common.``SummaryWriterDummy`(log_dir)

  Bases: `object`

  `add_scalar`(*args, **kwargs)
  `flush`()

`archai.common.common.``clean_ensure_expdir`(conf: Optional[archai.common.config.Config], clean_dir: bool, ensure_dir: bool) → None

`archai.common.common.``common_init`(config_filepath: Optional[str] = None, param_args: list = [], use_args=True, clean_expdir=False) → archai.common.config.Config

`archai.common.common.``create_conf`(config_filepath: Optional[str] = None, param_args: list = [], use_args=True) → archai.common.config.Config

`archai.common.common.``create_dirs`(conf: archai.common.config.Config, clean_expdir: bool) → Optional[str]

`archai.common.common.``create_logger`(conf: archai.common.config.Config)

`archai.common.common.``create_tb_writer`(conf: archai.common.config.Config, is_master=True) → Union[archai.common.common.SummaryWriterDummy, torch.utils.tensorboard.writer.SummaryWriter]

`archai.common.common.``expdir_abspath`(path: str, create=False) → str

  Returns full path for given relative path within experiment directory.

`archai.common.common.``get_conf`(conf: Optional[archai.common.config.Config] = None) → archai.common.config.Config

`archai.common.common.``get_conf_common`(conf: Optional[archai.common.config.Config] = None) → archai.common.config.Config

`archai.common.common.``get_conf_dataset`(conf: Optional[archai.common.config.Config] = None) → archai.common.config.Config

`archai.common.common.``get_datadir`(conf: Optional[archai.common.config.Config] = None) → Optional[str]

`archai.common.common.``get_expdir`(conf: Optional[archai.common.config.Config] = None) → Optional[str]

`archai.common.common.``get_experiment_name`(conf: Optional[archai.common.config.Config] = None) → str

`archai.common.common.``get_logger`() → archai.common.ordereddict_logger.OrderedDictLogger

`archai.common.common.``get_state`() → archai.common.common.CommonState

`archai.common.common.``get_tb_writer`() → Union[archai.common.common.SummaryWriterDummy, torch.utils.tensorboard.writer.SummaryWriter]

`archai.common.common.``init_from`(state: archai.common.common.CommonState, recreate_logger=True) → None

`archai.common.common.``on_app_exit`()

`archai.common.common.``update_envvars`(conf) → None

## archai.common.config module

class `archai.common.config.``Config`(config_filepath: Optional[str] = None, app_desc: Optional[str] = None, use_args=False, param_args: Sequence = [], resolve_redirects=True)

  Bases: `collections.UserDict`

  static `get_inst`() → archai.common.config.Config
  `get_val`(key, default_val)
  static `set_inst`(instance: archai.common.config.Config) → None
  `to_dict`() → dict

`archai.common.config.``deep_update`(d: collections.abc.MutableMapping, u: collections.abc.Mapping, create_map: Callable[], collections.abc.MutableMapping]) → collections.abc.MutableMapping

## archai.common.metrics module

class `archai.common.metrics.``Accumulator`

  Bases: `object`

  `add`(key, value)
  `add_dict`(dict)
  `get_dict`()
  `items`()

class `archai.common.metrics.``EpochMetrics`(index: int)

  Bases: `object`

  `duration`()
  `post_epoch`(val_metrics: Optional[archai.common.metrics.Metrics], lr: float)
  `post_step`(top1: float, top5: float, loss: float, batch: int)
  `pre_epoch`(lr: float)
  `pre_step`()

class `archai.common.metrics.``Metrics`(title: str, apex: Optional[archai.common.apex_utils.ApexUtils], logger_freq: int = 50)

  Bases: `object`

  Record top1, top5, loss metrics, track best so far.

  There are 3 levels of metrics: 1. Run level - these for the one call of ‘fit’, example, best top1 2. Epoch level - these are the averages maintained top1, top5, loss 3. Step level - these are for every step in epoch

  The pre_run must be called before fit call which will reset all metrics. Similarly pre_epoch will reset running averages and pre_step will reset step level metrics like average step time.

  The post_step will simply update the running averages while post_epoch updates best we have seen for each epoch.

  `best_train_top1`() → float
  `best_val_top1`() → float
  `cur_epoch`() → archai.common.metrics.EpochMetrics
  `epochs`() → int

    Returns epochs recorded so far

  `is_dist`() → bool
  `load_state_dict`(state_dict: dict) → None
  `post_epoch`(val_metrics: Optional[Metrics], lr: float = nan)
  `post_run`() → None
  `post_step`(x: torch.Tensor, y: torch.Tensor, logits: torch.Tensor, loss: torch.Tensor, steps: int) → None
  `pre_epoch`(lr: float = nan) → None
  `pre_run`() → None
  `pre_step`(x: torch.Tensor, y: torch.Tensor)
  `reduce_max`(val)
  `reduce_mean`(val)
  `reduce_min`(val)
  `reduce_sum`(val)
  `save`(filepath: str) → Optional[str]
  `state_dict`() → Mapping

class `archai.common.metrics.``RunMetrics`

  Bases: `object`

  `add_epoch`() → archai.common.metrics.EpochMetrics
  `best_epoch`() → Tuple[archai.common.metrics.EpochMetrics, Optional[archai.common.metrics.EpochMetrics]]
  `cur_epoch`() → archai.common.metrics.EpochMetrics
  `duration`()
  `epoch_time_avg`()
  `post_run`()
  `pre_run`()
  `step_time_avg`()

## archai.common.ml_losses module

class `archai.common.ml_losses.``LabelSmoothing`(smoothing=0.0)

  Bases: `torch.nn.modules.module.Module`

  NLL loss with label smoothing.

  `forward`(x, target)

    Defines the computation performed at every call.

    Should be overridden by all subclasses.

    Note

    Although the recipe for forward pass needs to be defined within this function, one should call the `Module` instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class `archai.common.ml_losses.``NLLMultiLabelSmooth`(smoothing=0.0)

  Bases: `torch.nn.modules.module.Module`

  According to NVidia code, this should be used with mixup?

  `forward`(x, target)

    Defines the computation performed at every call.

    Should be overridden by all subclasses.

    Note

    Although the recipe for forward pass needs to be defined within this function, one should call the `Module` instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class `archai.common.ml_losses.``SmoothCrossEntropyLoss`(weight=None, reduction='mean', smoothing=0.0)

  Bases: `torch.nn.modules.loss._WeightedLoss`

  `forward`(inputs, targets)

    Defines the computation performed at every call.

    Should be overridden by all subclasses.

    Note

    Although the recipe for forward pass needs to be defined within this function, one should call the `Module` instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

## archai.common.ml_utils module

`archai.common.ml_utils.``accuracy`(output, target, topk=1)

  Computes the precision@k for the specified values of k

`archai.common.ml_utils.``channel_norm`(dataset, channel_dim=1) → tuple

`archai.common.ml_utils.``create_lr_scheduler`(conf_lrs: archai.common.config.Config, epochs: int, optimizer: torch.optim.optimizer.Optimizer, steps_per_epoch: Optional[int]) → Tuple[Optional[torch.optim.lr_scheduler._LRScheduler], bool]

`archai.common.ml_utils.``create_optimizer`(conf_opt: archai.common.config.Config, params) → torch.optim.optimizer.Optimizer

`archai.common.ml_utils.``drop_path_`(x, drop_prob, training)

`archai.common.ml_utils.``ensure_pytorch_ver`(min_ver: str, error_msg: str) → bool

`archai.common.ml_utils.``get_lossfn`(conf_lossfn: archai.common.config.Config) → torch.nn.modules.loss._Loss

`archai.common.ml_utils.``get_optim_lr`(optimizer: torch.optim.optimizer.Optimizer) → float

`archai.common.ml_utils.``join_chunks`(chunks: List[torch.Tensor]) → torch.Tensor

  If batch was divided in chunks, this functions joins them again

`archai.common.ml_utils.``load_model`(model, model_path)

`archai.common.ml_utils.``param_size`(module: torch.nn.modules.module.Module)

  count all parameters excluding auxiliary

`archai.common.ml_utils.``save_model`(model, model_path)

`archai.common.ml_utils.``set_optim_lr`(optimizer: torch.optim.optimizer.Optimizer, lr: float) → None

## archai.common.model_summary module

`archai.common.model_summary.``get_all_tensor_stats`(o)

`archai.common.model_summary.``get_shape`(o)

`archai.common.model_summary.``get_tensor_stat`(tensor)

`archai.common.model_summary.``is_scaler`(o)

`archai.common.model_summary.``summary`(model, input_size)

`archai.common.model_summary.``summary_string`(model, input_size, dtype=torch.float32)

## archai.common.multi_optim module

class `archai.common.multi_optim.``MultiOptim`

  Bases: `object`

  `append`(optim_sched: archai.common.multi_optim.OptimSched) → None
  `epoch`(epoch: Optional[int] = None) → None
  `get_lr`(optim_index: int, param_index: int) → float
  `load_state_dict`(state_dict: dict) → None
  `state_dict`() → dict
  `step`() → None
  `zero_grad`() → None

class `archai.common.multi_optim.``OptimSched`(optim: torch.optim.optimizer.Optimizer, sched: Optional[torch.optim.lr_scheduler._LRScheduler], sched_on_epoch: Optional[bool])

  Bases: `object`

  Holds the optimizer and scheduler

## archai.common.ordereddict_logger module

class `archai.common.ordereddict_logger.``OrderedDictLogger`(filepath: Optional[str], logger: Optional[logging.Logger], save_delay: Optional[float] = 30.0, yaml_log=True)

  Bases: `object`

  The purpose of the structured logging is to store logs as key value pair. However, when you have loop and sub routine calls, what you need is hierarchical dictionaries where the value for a key could be a dictionary. The idea is that you set one of the nodes in tree as current node and start logging your values. You can then use pushd to create and go to child node and popd to come back to parent. To implement this mechanism we use two main variables: _stack allows us to push each node on stack when pushd is called. The node is OrderedDictionary. As a convinience, we let specify child path in pushd in which case child hierarchy is created and current node will be set to the last node in specified path. When popd is called, we go back to original parent instead of parent of current node. To implement this we use _paths variable which stores subpath when each pushd call was made.

  `close`() → None
  `debug`(dict: Union[Mapping, str], level: Optional[int] = 10, exists_ok=False) → None
  `info`(dict: Union[Mapping, str], level: Optional[int] = 20, exists_ok=False) → None
  `load`(filepath: str) → None
  `path`() → str
  `popd`()
  `pushd`(*keys: Any) → archai.common.ordereddict_logger.OrderedDictLogger
  `reset`(filepath: Optional[str], logger: Optional[logging.Logger], save_delay: Optional[float] = 30.0, load_existing_file=False, backup_existing_file=True, yaml_log=True) → None
  `save`(filepath: Optional[str] = None) → None
  `warn`(dict: Union[Mapping, str], level: Optional[int] = 30, exists_ok=False) → None

## archai.common.stopwatch module

class `archai.common.stopwatch.``StopWatch`

  Bases: `object`

  `clear`(tag=None)
  static `get`() → archai.common.stopwatch.StopWatch
  `get_elapsed`(tag=None)
  `keys`()
  `pause`(tag=None)
  static `set`(instance: archai.common.stopwatch.StopWatch) → None
  `start`(tag=None)
  `stopwatch`: StopWatch = None

## archai.common.tester module

class `archai.common.tester.``Tester`(conf_val: archai.common.config.Config, model: torch.nn.modules.module.Module, apex: archai.common.apex_utils.ApexUtils)

  Bases: `overrides.enforce.EnforceOverrides`

  `get_device`()
  `get_metrics`() → Optional[archai.common.metrics.Metrics]
  `load_state_dict`(state_dict: dict) → None
  `state_dict`() → dict
  `test`(test_dl: torch.utils.data.dataloader.DataLoader) → archai.common.metrics.Metrics

## archai.common.timing module

class `archai.common.timing.``MeasureBlockTime`(name: str, no_print=True, disable_gc=False)

  Bases: `object`

  `cur_elapsed`() → float

`archai.common.timing.``MeasureTime`(f_py=None, no_print=True, disable_gc=False, name: Optional[str] = None)

`archai.common.timing.``add_timing`(name: str, elapsed: float, no_print=True) → runstats.fast.Statistics

`archai.common.timing.``clear_timings`() → None

`archai.common.timing.``get_all_timings`() → Dict[str, runstats.fast.Statistics]

`archai.common.timing.``get_last`(name: str) → float

`archai.common.timing.``get_timing`(name: str) → runstats.fast.Statistics

`archai.common.timing.``print_all_timings`() → None

`archai.common.timing.``print_timing`(name: str) → None

## archai.common.trainer module

class `archai.common.trainer.``Trainer`(conf_train: archai.common.config.Config, model: torch.nn.modules.module.Module, checkpoint: Optional[archai.common.checkpoint.CheckPoint])

  Bases: `overrides.enforce.EnforceOverrides`

  `compute_loss`(lossfn: Callable, y: torch.Tensor, logits: torch.Tensor, aux_weight: float, aux_logits: Optional[torch.Tensor]) → torch.Tensor
  `create_multi_optim`(train_len: int) → archai.common.multi_optim.MultiOptim
  `create_optimizer`(conf_optim: archai.common.config.Config, params) → torch.optim.optimizer.Optimizer
  `create_scheduler`(conf_sched: archai.common.config.Config, optim: torch.optim.optimizer.Optimizer, steps_per_epoch: int) → Tuple[Optional[torch.optim.lr_scheduler._LRScheduler], bool]
  `epoch`() → int
  `fit`(train_dl: torch.utils.data.dataloader.DataLoader, val_dl: Optional[torch.utils.data.dataloader.DataLoader]) → archai.common.metrics.Metrics
  `get_device`()
  `get_metrics`() → archai.common.metrics.Metrics
  `get_optimizer`(index=0) → torch.optim.optimizer.Optimizer
  `get_scheduler`(index=0) → Optional[torch.optim.lr_scheduler._LRScheduler]
  `post_epoch`(train_dl: torch.utils.data.dataloader.DataLoader, val_dl: Optional[torch.utils.data.dataloader.DataLoader]) → None
  `post_fit`(train_dl: torch.utils.data.dataloader.DataLoader, val_dl: Optional[torch.utils.data.dataloader.DataLoader]) → None
  `post_step`(x: torch.Tensor, y: torch.Tensor, logits: torch.Tensor, loss: torch.Tensor, steps: int) → None
  `pre_epoch`(train_dl: torch.utils.data.dataloader.DataLoader, val_dl: Optional[torch.utils.data.dataloader.DataLoader]) → None
  `pre_fit`(train_dl: torch.utils.data.dataloader.DataLoader, val_dl: Optional[torch.utils.data.dataloader.DataLoader]) → None
  `pre_step`(x: torch.Tensor, y: torch.Tensor) → None
  `restore_checkpoint`() → None
  `update_checkpoint`(checkpoint: archai.common.checkpoint.CheckPoint) → None

## archai.common.utils module

class `archai.common.utils.``AverageMeter`

  Bases: `object`

  `reset`()
  `update`(val, n=1)

`archai.common.utils.``append_csv_file`(filepath: str, new_row: List[Tuple[str, Any]], delimiter='\t')

`archai.common.utils.``append_to_filename`(filepath: str, name_suffix: str, new_ext: Optional[str] = None) → str

  Returns ‘/a/b/c/h.f’ for filepath=’/a/b/c/d.e.f’, new_name=’h’

`archai.common.utils.``change_filepath_ext`(filepath: str, new_ext: str) → str

  Returns ‘/a/b/c/d.e.g’ for filepath=’/a/b/c/d.e.f’, new_ext=’.g’

`archai.common.utils.``change_filepath_name`(filepath: str, new_name: str, new_ext: Optional[str] = None) → str

  Returns ‘/a/b/c/h.f’ for filepath=’/a/b/c/d.e.f’, new_name=’h’

`archai.common.utils.``copy_dir`(src_dir: str, dest_dir: str, use_shutil: bool = True) → None

`archai.common.utils.``copy_file`(src_file: str, dest_dir_or_file: str, preserve_metadata=False, use_shutil: bool = True) → str

`archai.common.utils.``copy_file_basic`(src_file: str, dest_dir_or_file: str) → str

`archai.common.utils.``create_logger`(filepath: Optional[str] = None, name: Optional[str] = None, level=20, enable_stdout=True) → logging.Logger

`archai.common.utils.``cuda_device_names`() → str

`archai.common.utils.``deep_comp`(o1: Any, o2: Any) → bool

`archai.common.utils.``deep_update`(d: MutableMapping, u: Mapping, map_type: Type[MutableMapping] = <class 'dict'>) → MutableMapping

`archai.common.utils.``dir_downloads`() → str

`archai.common.utils.``download_and_extract_tar`(url, download_root, extract_root=None, filename=None, md5=None, **kwargs)

`archai.common.utils.``exec_shell_command`(command: str, print_command=True) → None

`archai.common.utils.``extract_tar`(src, dest=None, gzip=None, delete=False)

`archai.common.utils.``filepath_ext`(filepath: str) → str

  Returns ‘d.e.f’ for ‘/a/b/c/d.e.f’

`archai.common.utils.``filepath_name_ext`(filepath: str) → str

  Returns ‘.f’ for ‘/a/b/c/d.e.f’

`archai.common.utils.``filepath_name_only`(filepath: str) → str

  Returns ‘d.e’ for ‘/a/b/c/d.e.f’

`archai.common.utils.``filepath_without_ext`(filepath: str) → str

  Returns ‘/a/b/c/d.e’ for ‘/a/b/c/d.e.f’

`archai.common.utils.``first_or_default`(it: Iterable, default=None)

`archai.common.utils.``fmt`(val: Any) → str

`archai.common.utils.``full_path`(path: str, create=False) → str

`archai.common.utils.``has_method`(o, name)

`archai.common.utils.``is_debugging`() → bool

`archai.common.utils.``is_main_process`() → bool

  Returns True if this process was started as main process instead of child process during multiprocessing

`archai.common.utils.``load_state_dict`(val: Any, state_dict: Mapping) → None

`archai.common.utils.``main_process_pid`() → int

`archai.common.utils.``process_name`() → str

`archai.common.utils.``read_string`(filepath: str) → str

`archai.common.utils.``setup_cuda`(seed: Union[float, int], local_rank: int)

`archai.common.utils.``state_dict`(val) → Mapping

`archai.common.utils.``write_string`(filepath: str, content: str) → None

`archai.common.utils.``zero_file`(filepath) → None

  Creates or truncates existing file

`archai.common.utils.``zip_eq`(*iterables)

## archai.common.warmup_scheduler module

class `archai.common.warmup_scheduler.``GradualWarmupScheduler`(optimizer, multiplier, total_epoch, after_scheduler=None)

  Bases: `torch.optim.lr_scheduler._LRScheduler`

  Gradually warm-up(increasing) learning rate in optimizer. Proposed in ‘Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour’.

  Args:

    optimizer (Optimizer): Wrapped optimizer. multiplier: target learning rate = base lr * multiplier if multiplier > 1.0. if multiplier = 1.0, lr starts from 0 and ends up with the base_lr. total_epoch: target learning rate is reached at total_epoch, gradually after_scheduler: after target_epoch, use this scheduler(eg. ReduceLROnPlateau)

  `get_lr`()
  `step`(epoch=None, metrics=None)
  `step_ReduceLROnPlateau`(metrics, epoch=None)

## archai.common.yaml_utils module

`archai.common.yaml_utils.``is_proper_path`(path: str) → bool

`archai.common.yaml_utils.``resolve_all`(root_d: MutableMapping)

## Module contents