archai.common package¶
Submodules¶
archai.common.apex_utils module¶
-
class
archai.common.apex_utils.
ApexUtils
(apex_config: archai.common.config.Config, logger: Optional[archai.common.ordereddict_logger.OrderedDictLogger])[source]¶ Bases:
object
-
backward
(loss: torch.Tensor, multi_optim: archai.common.multi_optim.MultiOptim) → None[source]¶
-
clip_grad
(clip: float, model: torch.nn.modules.module.Module, multi_optim: archai.common.multi_optim.MultiOptim) → None[source]¶
-
to_amp
(model: torch.nn.modules.module.Module, multi_optim: archai.common.multi_optim.MultiOptim, batch_size: int) → torch.nn.modules.module.Module[source]¶
-
archai.common.atomic_file_handler module¶
-
class
archai.common.atomic_file_handler.
AtomicFileHandler
(filename, encoding=None, save_delay=30.0)[source]¶ Bases:
logging.Handler
This class opens and writes entire file instead of appending one line at a time
-
close
()[source]¶ Tidy up any resources used by the handler.
This version removes the handler from an internal map of handlers, _handlers, which is used for handler lookup by name. Subclasses should ensure that this gets called from overridden close() methods.
-
emit
(record)[source]¶ Do whatever it takes to actually log the specified logging record.
This version is intended to be implemented by subclasses and so raises a NotImplementedError.
-
flush
()[source]¶ Ensure all logging output has been flushed.
This version does nothing and is intended to be implemented by subclasses.
-
terminator
= '\n'¶
-
archai.common.checkpoint module¶
-
class
archai.common.checkpoint.
CheckPoint
(conf_checkpoint: archai.common.config.Config, load_existing: bool)[source]¶ Bases:
collections.UserDict
Callback based checkpoint model.
Start new checkpoint by calling new() and save it by calling commit(). This class is also dictionary. Items that needs be saved can be done so by setting key, value pairs after new(). As any dictionary key is set, checkpoint becomes dirty. On commit(), dictionary is saved and emptied. Invariant: checkpoint remains dirty until commit() is called.
archai.common.cocob module¶
The code below is directly from https://raw.githubusercontent.com/IssamLaradji/sls/master/others/cocob.py Two coin betting optimization algorithms are implemented here : Cocob Backprop: https://arxiv.org/pdf/1705.07795.pdf Cocob through Ons: https://arxiv.org/pdf/1705.07795.pdf both of which do not require any learning rates and yet have optimal convergence gauarantees for non-smooth convex functions.
Cocob-Ons is an experimental variation from paper. Please don’t use it yet.
Please check http://francesco.orabona.com/papers/slides_cocob.pdf for simple explanation for going from coin betting game to convex optimization. Both algorithms are similar except the coin betting strategy used.
-
class
archai.common.cocob.
CocobBackprop
(params, alpha=100.0, eps=1e-08)[source]¶ Bases:
torch.optim.optimizer.Optimizer
Implements Cocob-Backprop .
It has been proposed in Training Deep Networks without Learning Rates Through Coin Betting.
- Arguments:
- params (iterable): iterable of parameters to optimize or dicts defining
parameter groups
- alpha (float, optional): positive number to adjust betting fraction.
Theoretical convergence gauarantee does not depend on choice of alpha (default: 100.0)
-
class
archai.common.cocob.
CocobOns
(params, eps=1e-08)[source]¶ Bases:
torch.optim.optimizer.Optimizer
Implements Coin-Betting through ONS .
It has been proposed in Black-Box Reductions for Parameter-free Online Learning in Banach Spaces.
Cocob-Ons is an experimental variation from the paper. Do not use it yet.
- Arguments:
- params (iterable): iterable of parameters to optimize or dicts defining
parameter groups
- eps (float, optional): positive initial wealth for betting algorithm.
Theoretical convergence gauarantee does not depend on choice of eps (default: 1e-8)
archai.common.common module¶
-
archai.common.common.
clean_ensure_expdir
(conf: Optional[archai.common.config.Config], clean_dir: bool, ensure_dir: bool) → None[source]¶
-
archai.common.common.
common_init
(config_filepath: Optional[str] = None, param_args: list = [], use_args=True, clean_expdir=False) → archai.common.config.Config[source]¶
-
archai.common.common.
create_conf
(config_filepath: Optional[str] = None, param_args: list = [], use_args=True) → archai.common.config.Config[source]¶
-
archai.common.common.
create_dirs
(conf: archai.common.config.Config, clean_expdir: bool) → Optional[str][source]¶
-
archai.common.common.
create_logger
(conf: archai.common.config.Config)[source]¶
-
archai.common.common.
create_tb_writer
(conf: archai.common.config.Config, is_master=True) → Union[archai.common.common.SummaryWriterDummy, torch.utils.tensorboard.writer.SummaryWriter][source]¶
-
archai.common.common.
expdir_abspath
(path: str, create=False) → str[source]¶ Returns full path for given relative path within experiment directory.
-
archai.common.common.
get_conf
(conf: Optional[archai.common.config.Config] = None) → archai.common.config.Config[source]¶
-
archai.common.common.
get_conf_common
(conf: Optional[archai.common.config.Config] = None) → archai.common.config.Config[source]¶
-
archai.common.common.
get_conf_dataset
(conf: Optional[archai.common.config.Config] = None) → archai.common.config.Config[source]¶
-
archai.common.common.
get_datadir
(conf: Optional[archai.common.config.Config] = None) → Optional[str][source]¶
-
archai.common.common.
get_expdir
(conf: Optional[archai.common.config.Config] = None) → Optional[str][source]¶
-
archai.common.common.
get_experiment_name
(conf: Optional[archai.common.config.Config] = None) → str[source]¶
-
archai.common.common.
get_logger
() → archai.common.ordereddict_logger.OrderedDictLogger[source]¶
-
archai.common.common.
get_state
() → archai.common.common.CommonState[source]¶
-
archai.common.common.
get_tb_writer
() → Union[archai.common.common.SummaryWriterDummy, torch.utils.tensorboard.writer.SummaryWriter][source]¶
-
archai.common.common.
init_from
(state: archai.common.common.CommonState, recreate_logger=True) → None[source]¶
archai.common.config module¶
-
class
archai.common.config.
Config
(config_filepath: Optional[str] = None, app_desc: Optional[str] = None, use_args=False, param_args: Sequence = [], resolve_redirects=True)[source]¶ Bases:
collections.UserDict
-
static
get_inst
() → archai.common.config.Config[source]¶
-
static
set_inst
(instance: archai.common.config.Config) → None[source]¶
-
static
archai.common.metrics module¶
-
class
archai.common.metrics.
EpochMetrics
(index: int)[source]¶ Bases:
object
-
post_epoch
(val_metrics: Optional[archai.common.metrics.Metrics], lr: float)[source]¶
-
-
class
archai.common.metrics.
Metrics
(title: str, apex: Optional[archai.common.apex_utils.ApexUtils], logger_freq: int = 50)[source]¶ Bases:
object
Record top1, top5, loss metrics, track best so far.
There are 3 levels of metrics: 1. Run level - these for the one call of ‘fit’, example, best top1 2. Epoch level - these are the averages maintained top1, top5, loss 3. Step level - these are for every step in epoch
The pre_run must be called before fit call which will reset all metrics. Similarly pre_epoch will reset running averages and pre_step will reset step level metrics like average step time.
The post_step will simply update the running averages while post_epoch updates best we have seen for each epoch.
-
cur_epoch
() → archai.common.metrics.EpochMetrics[source]¶
-
-
class
archai.common.metrics.
RunMetrics
[source]¶ Bases:
object
-
add_epoch
() → archai.common.metrics.EpochMetrics[source]¶
-
best_epoch
() → Tuple[archai.common.metrics.EpochMetrics, Optional[archai.common.metrics.EpochMetrics]][source]¶
-
cur_epoch
() → archai.common.metrics.EpochMetrics[source]¶
-
archai.common.ml_losses module¶
-
class
archai.common.ml_losses.
LabelSmoothing
(smoothing=0.0)[source]¶ Bases:
torch.nn.modules.module.Module
NLL loss with label smoothing.
-
forward
(x, target)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
archai.common.ml_losses.
NLLMultiLabelSmooth
(smoothing=0.0)[source]¶ Bases:
torch.nn.modules.module.Module
According to NVidia code, this should be used with mixup?
-
forward
(x, target)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
-
class
archai.common.ml_losses.
SmoothCrossEntropyLoss
(weight=None, reduction='mean', smoothing=0.0)[source]¶ Bases:
torch.nn.modules.loss._WeightedLoss
-
forward
(inputs, targets)[source]¶ Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
-
archai.common.ml_utils module¶
-
archai.common.ml_utils.
accuracy
(output, target, topk=1)[source]¶ Computes the precision@k for the specified values of k
-
archai.common.ml_utils.
create_lr_scheduler
(conf_lrs: archai.common.config.Config, epochs: int, optimizer: torch.optim.optimizer.Optimizer, steps_per_epoch: Optional[int]) → Tuple[Optional[torch.optim.lr_scheduler._LRScheduler], bool][source]¶
-
archai.common.ml_utils.
create_optimizer
(conf_opt: archai.common.config.Config, params) → torch.optim.optimizer.Optimizer[source]¶
-
archai.common.ml_utils.
get_lossfn
(conf_lossfn: archai.common.config.Config) → torch.nn.modules.loss._Loss[source]¶
-
archai.common.ml_utils.
join_chunks
(chunks: List[torch.Tensor]) → torch.Tensor[source]¶ If batch was divided in chunks, this functions joins them again
archai.common.model_summary module¶
archai.common.multi_optim module¶
-
class
archai.common.multi_optim.
MultiOptim
[source]¶ Bases:
object
-
append
(optim_sched: archai.common.multi_optim.OptimSched) → None[source]¶
-
archai.common.ordereddict_logger module¶
-
class
archai.common.ordereddict_logger.
OrderedDictLogger
(filepath: Optional[str], logger: Optional[logging.Logger], save_delay: Optional[float] = 30.0, yaml_log=True)[source]¶ Bases:
object
The purpose of the structured logging is to store logs as key value pair. However, when you have loop and sub routine calls, what you need is hierarchical dictionaries where the value for a key could be a dictionary. The idea is that you set one of the nodes in tree as current node and start logging your values. You can then use pushd to create and go to child node and popd to come back to parent. To implement this mechanism we use two main variables: _stack allows us to push each node on stack when pushd is called. The node is OrderedDictionary. As a convinience, we let specify child path in pushd in which case child hierarchy is created and current node will be set to the last node in specified path. When popd is called, we go back to original parent instead of parent of current node. To implement this we use _paths variable which stores subpath when each pushd call was made.
-
pushd
(*keys: Any) → archai.common.ordereddict_logger.OrderedDictLogger[source]¶
-
archai.common.stopwatch module¶
-
class
archai.common.stopwatch.
StopWatch
[source]¶ Bases:
object
-
static
get
() → archai.common.stopwatch.StopWatch[source]¶
-
static
set
(instance: archai.common.stopwatch.StopWatch) → None[source]¶
-
stopwatch
: StopWatch = None¶
-
static
archai.common.tester module¶
-
class
archai.common.tester.
Tester
(conf_val: archai.common.config.Config, model: torch.nn.modules.module.Module, apex: archai.common.apex_utils.ApexUtils)[source]¶ Bases:
overrides.enforce.EnforceOverrides
-
get_metrics
() → Optional[archai.common.metrics.Metrics][source]¶
-
test
(test_dl: torch.utils.data.dataloader.DataLoader) → archai.common.metrics.Metrics[source]¶
-
archai.common.timing module¶
-
class
archai.common.timing.
MeasureBlockTime
(name: str, no_print=True, disable_gc=False)[source]¶ Bases:
object
-
archai.common.timing.
MeasureTime
(f_py=None, no_print=True, disable_gc=False, name: Optional[str] = None)[source]¶
archai.common.trainer module¶
-
class
archai.common.trainer.
Trainer
(conf_train: archai.common.config.Config, model: torch.nn.modules.module.Module, checkpoint: Optional[archai.common.checkpoint.CheckPoint])[source]¶ Bases:
overrides.enforce.EnforceOverrides
-
compute_loss
(lossfn: Callable, y: torch.Tensor, logits: torch.Tensor, aux_weight: float, aux_logits: Optional[torch.Tensor]) → torch.Tensor[source]¶
-
create_multi_optim
(train_len: int) → archai.common.multi_optim.MultiOptim[source]¶
-
create_optimizer
(conf_optim: archai.common.config.Config, params) → torch.optim.optimizer.Optimizer[source]¶
-
create_scheduler
(conf_sched: archai.common.config.Config, optim: torch.optim.optimizer.Optimizer, steps_per_epoch: int) → Tuple[Optional[torch.optim.lr_scheduler._LRScheduler], bool][source]¶
-
fit
(train_dl: torch.utils.data.dataloader.DataLoader, val_dl: Optional[torch.utils.data.dataloader.DataLoader]) → archai.common.metrics.Metrics[source]¶
-
get_metrics
() → archai.common.metrics.Metrics[source]¶
-
post_epoch
(train_dl: torch.utils.data.dataloader.DataLoader, val_dl: Optional[torch.utils.data.dataloader.DataLoader]) → None[source]¶
-
post_fit
(train_dl: torch.utils.data.dataloader.DataLoader, val_dl: Optional[torch.utils.data.dataloader.DataLoader]) → None[source]¶
-
post_step
(x: torch.Tensor, y: torch.Tensor, logits: torch.Tensor, loss: torch.Tensor, steps: int) → None[source]¶
-
pre_epoch
(train_dl: torch.utils.data.dataloader.DataLoader, val_dl: Optional[torch.utils.data.dataloader.DataLoader]) → None[source]¶
-
pre_fit
(train_dl: torch.utils.data.dataloader.DataLoader, val_dl: Optional[torch.utils.data.dataloader.DataLoader]) → None[source]¶
-
update_checkpoint
(checkpoint: archai.common.checkpoint.CheckPoint) → None[source]¶
-
archai.common.utils module¶
-
archai.common.utils.
append_csv_file
(filepath: str, new_row: List[Tuple[str, Any]], delimiter='\t')[source]¶
-
archai.common.utils.
append_to_filename
(filepath: str, name_suffix: str, new_ext: Optional[str] = None) → str[source]¶ Returns ‘/a/b/c/h.f’ for filepath=’/a/b/c/d.e.f’, new_name=’h’
-
archai.common.utils.
change_filepath_ext
(filepath: str, new_ext: str) → str[source]¶ Returns ‘/a/b/c/d.e.g’ for filepath=’/a/b/c/d.e.f’, new_ext=’.g’
-
archai.common.utils.
change_filepath_name
(filepath: str, new_name: str, new_ext: Optional[str] = None) → str[source]¶ Returns ‘/a/b/c/h.f’ for filepath=’/a/b/c/d.e.f’, new_name=’h’
-
archai.common.utils.
copy_file
(src_file: str, dest_dir_or_file: str, preserve_metadata=False, use_shutil: bool = True) → str[source]¶
-
archai.common.utils.
create_logger
(filepath: Optional[str] = None, name: Optional[str] = None, level=20, enable_stdout=True) → logging.Logger[source]¶
-
archai.common.utils.
deep_update
(d: MutableMapping, u: Mapping, map_type: Type[MutableMapping] = <class 'dict'>) → MutableMapping[source]¶
-
archai.common.utils.
download_and_extract_tar
(url, download_root, extract_root=None, filename=None, md5=None, **kwargs)[source]¶
-
archai.common.utils.
filepath_name_only
(filepath: str) → str[source]¶ Returns ‘d.e’ for ‘/a/b/c/d.e.f’
-
archai.common.utils.
filepath_without_ext
(filepath: str) → str[source]¶ Returns ‘/a/b/c/d.e’ for ‘/a/b/c/d.e.f’
archai.common.warmup_scheduler module¶
-
class
archai.common.warmup_scheduler.
GradualWarmupScheduler
(optimizer, multiplier, total_epoch, after_scheduler=None)[source]¶ Bases:
torch.optim.lr_scheduler._LRScheduler
Gradually warm-up(increasing) learning rate in optimizer. Proposed in ‘Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour’.
- Args:
optimizer (Optimizer): Wrapped optimizer. multiplier: target learning rate = base lr * multiplier if multiplier > 1.0. if multiplier = 1.0, lr starts from 0 and ends up with the base_lr. total_epoch: target learning rate is reached at total_epoch, gradually after_scheduler: after target_epoch, use this scheduler(eg. ReduceLROnPlateau)