archai.common package

Submodules

archai.common.apex_utils module

class archai.common.apex_utils.ApexUtils(apex_config: archai.common.config.Config, logger: Optional[archai.common.ordereddict_logger.OrderedDictLogger])[source]

Bases: object

backward(loss: torch.Tensor, multi_optim: archai.common.multi_optim.MultiOptim) → None[source]
barrier() → None[source]
clip_grad(clip: float, model: torch.nn.modules.module.Module, multi_optim: archai.common.multi_optim.MultiOptim) → None[source]
is_dist() → bool[source]
is_master() → bool[source]
is_mixed() → bool[source]
is_ray() → bool[source]
load_state_dict(state_dict)[source]
reduce(val, op='mean')[source]
state_dict()[source]
sync_devices() → None[source]
to_amp(model: torch.nn.modules.module.Module, multi_optim: archai.common.multi_optim.MultiOptim, batch_size: int) → torch.nn.modules.module.Module[source]

archai.common.atomic_file_handler module

class archai.common.atomic_file_handler.AtomicFileHandler(filename, encoding=None, save_delay=30.0)[source]

Bases: logging.Handler

This class opens and writes entire file instead of appending one line at a time

close()[source]

Tidy up any resources used by the handler.

This version removes the handler from an internal map of handlers, _handlers, which is used for handler lookup by name. Subclasses should ensure that this gets called from overridden close() methods.

emit(record)[source]

Do whatever it takes to actually log the specified logging record.

This version is intended to be implemented by subclasses and so raises a NotImplementedError.

flush()[source]

Ensure all logging output has been flushed.

This version does nothing and is intended to be implemented by subclasses.

terminator = '\n'

archai.common.checkpoint module

class archai.common.checkpoint.CheckPoint(conf_checkpoint: archai.common.config.Config, load_existing: bool)[source]

Bases: collections.UserDict

Callback based checkpoint model.

Start new checkpoint by calling new() and save it by calling commit(). This class is also dictionary. Items that needs be saved can be done so by setting key, value pairs after new(). As any dictionary key is set, checkpoint becomes dirty. On commit(), dictionary is saved and emptied. Invariant: checkpoint remains dirty until commit() is called.

commit() → None[source]
is_empty() → bool[source]
load_existing() → bool[source]
new(*kargs, **kvargs) → None[source]
subscribe(callback: Callable) → None[source]

archai.common.cocob module

The code below is directly from https://raw.githubusercontent.com/IssamLaradji/sls/master/others/cocob.py Two coin betting optimization algorithms are implemented here : Cocob Backprop: https://arxiv.org/pdf/1705.07795.pdf Cocob through Ons: https://arxiv.org/pdf/1705.07795.pdf both of which do not require any learning rates and yet have optimal convergence gauarantees for non-smooth convex functions.

Cocob-Ons is an experimental variation from paper. Please don’t use it yet.

Please check http://francesco.orabona.com/papers/slides_cocob.pdf for simple explanation for going from coin betting game to convex optimization. Both algorithms are similar except the coin betting strategy used.

class archai.common.cocob.CocobBackprop(params, alpha=100.0, eps=1e-08)[source]

Bases: torch.optim.optimizer.Optimizer

Implements Cocob-Backprop .

It has been proposed in Training Deep Networks without Learning Rates Through Coin Betting.

Arguments:
params (iterable): iterable of parameters to optimize or dicts defining

parameter groups

alpha (float, optional): positive number to adjust betting fraction.

Theoretical convergence gauarantee does not depend on choice of alpha (default: 100.0)

step(closure=None)[source]

Performs a single optimization step (parameter update).

Arguments:
closure (callable): A closure that reevaluates the model and

returns the loss. Optional for most optimizers.

class archai.common.cocob.CocobOns(params, eps=1e-08)[source]

Bases: torch.optim.optimizer.Optimizer

Implements Coin-Betting through ONS .

It has been proposed in Black-Box Reductions for Parameter-free Online Learning in Banach Spaces.

Cocob-Ons is an experimental variation from the paper. Do not use it yet.

Arguments:
params (iterable): iterable of parameters to optimize or dicts defining

parameter groups

eps (float, optional): positive initial wealth for betting algorithm.

Theoretical convergence gauarantee does not depend on choice of eps (default: 1e-8)

step(closure=None)[source]

Performs a single optimization step (parameter update).

Arguments:
closure (callable): A closure that reevaluates the model and

returns the loss. Optional for most optimizers.

archai.common.common module

class archai.common.common.CommonState[source]

Bases: object

class archai.common.common.SummaryWriterDummy(log_dir)[source]

Bases: object

add_scalar(*args, **kwargs)[source]
flush()[source]
archai.common.common.clean_ensure_expdir(conf: Optional[archai.common.config.Config], clean_dir: bool, ensure_dir: bool) → None[source]
archai.common.common.common_init(config_filepath: Optional[str] = None, param_args: list = [], use_args=True, clean_expdir=False)archai.common.config.Config[source]
archai.common.common.create_conf(config_filepath: Optional[str] = None, param_args: list = [], use_args=True)archai.common.config.Config[source]
archai.common.common.create_dirs(conf: archai.common.config.Config, clean_expdir: bool) → Optional[str][source]
archai.common.common.create_logger(conf: archai.common.config.Config)[source]
archai.common.common.create_tb_writer(conf: archai.common.config.Config, is_master=True) → Union[archai.common.common.SummaryWriterDummy, torch.utils.tensorboard.writer.SummaryWriter][source]
archai.common.common.expdir_abspath(path: str, create=False) → str[source]

Returns full path for given relative path within experiment directory.

archai.common.common.get_conf(conf: Optional[archai.common.config.Config] = None)archai.common.config.Config[source]
archai.common.common.get_conf_common(conf: Optional[archai.common.config.Config] = None)archai.common.config.Config[source]
archai.common.common.get_conf_dataset(conf: Optional[archai.common.config.Config] = None)archai.common.config.Config[source]
archai.common.common.get_datadir(conf: Optional[archai.common.config.Config] = None) → Optional[str][source]
archai.common.common.get_expdir(conf: Optional[archai.common.config.Config] = None) → Optional[str][source]
archai.common.common.get_experiment_name(conf: Optional[archai.common.config.Config] = None) → str[source]
archai.common.common.get_logger()archai.common.ordereddict_logger.OrderedDictLogger[source]
archai.common.common.get_state()archai.common.common.CommonState[source]
archai.common.common.get_tb_writer() → Union[archai.common.common.SummaryWriterDummy, torch.utils.tensorboard.writer.SummaryWriter][source]
archai.common.common.init_from(state: archai.common.common.CommonState, recreate_logger=True) → None[source]
archai.common.common.on_app_exit()[source]
archai.common.common.update_envvars(conf) → None[source]

archai.common.config module

class archai.common.config.Config(config_filepath: Optional[str] = None, app_desc: Optional[str] = None, use_args=False, param_args: Sequence = [], resolve_redirects=True)[source]

Bases: collections.UserDict

static get_inst()archai.common.config.Config[source]
get_val(key, default_val)[source]
static set_inst(instance: archai.common.config.Config) → None[source]
to_dict() → dict[source]
archai.common.config.deep_update(d: collections.abc.MutableMapping, u: collections.abc.Mapping, create_map: Callable[], collections.abc.MutableMapping]) → collections.abc.MutableMapping[source]

archai.common.metrics module

class archai.common.metrics.Accumulator[source]

Bases: object

add(key, value)[source]
add_dict(dict)[source]
get_dict()[source]
items()[source]
class archai.common.metrics.EpochMetrics(index: int)[source]

Bases: object

duration()[source]
post_epoch(val_metrics: Optional[archai.common.metrics.Metrics], lr: float)[source]
post_step(top1: float, top5: float, loss: float, batch: int)[source]
pre_epoch(lr: float)[source]
pre_step()[source]
class archai.common.metrics.Metrics(title: str, apex: Optional[archai.common.apex_utils.ApexUtils], logger_freq: int = 50)[source]

Bases: object

Record top1, top5, loss metrics, track best so far.

There are 3 levels of metrics: 1. Run level - these for the one call of ‘fit’, example, best top1 2. Epoch level - these are the averages maintained top1, top5, loss 3. Step level - these are for every step in epoch

The pre_run must be called before fit call which will reset all metrics. Similarly pre_epoch will reset running averages and pre_step will reset step level metrics like average step time.

The post_step will simply update the running averages while post_epoch updates best we have seen for each epoch.

best_train_top1() → float[source]
best_val_top1() → float[source]
cur_epoch()archai.common.metrics.EpochMetrics[source]
epochs() → int[source]

Returns epochs recorded so far

is_dist() → bool[source]
load_state_dict(state_dict: dict) → None[source]
post_epoch(val_metrics: Optional[Metrics], lr: float = nan)[source]
post_run() → None[source]
post_step(x: torch.Tensor, y: torch.Tensor, logits: torch.Tensor, loss: torch.Tensor, steps: int) → None[source]
pre_epoch(lr: float = nan) → None[source]
pre_run() → None[source]
pre_step(x: torch.Tensor, y: torch.Tensor)[source]
reduce_max(val)[source]
reduce_mean(val)[source]
reduce_min(val)[source]
reduce_sum(val)[source]
save(filepath: str) → Optional[str][source]
state_dict() → Mapping[source]
class archai.common.metrics.RunMetrics[source]

Bases: object

add_epoch()archai.common.metrics.EpochMetrics[source]
best_epoch() → Tuple[archai.common.metrics.EpochMetrics, Optional[archai.common.metrics.EpochMetrics]][source]
cur_epoch()archai.common.metrics.EpochMetrics[source]
duration()[source]
epoch_time_avg()[source]
post_run()[source]
pre_run()[source]
step_time_avg()[source]

archai.common.ml_losses module

class archai.common.ml_losses.LabelSmoothing(smoothing=0.0)[source]

Bases: torch.nn.modules.module.Module

NLL loss with label smoothing.

forward(x, target)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class archai.common.ml_losses.NLLMultiLabelSmooth(smoothing=0.0)[source]

Bases: torch.nn.modules.module.Module

According to NVidia code, this should be used with mixup?

forward(x, target)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class archai.common.ml_losses.SmoothCrossEntropyLoss(weight=None, reduction='mean', smoothing=0.0)[source]

Bases: torch.nn.modules.loss._WeightedLoss

forward(inputs, targets)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

archai.common.ml_utils module

archai.common.ml_utils.accuracy(output, target, topk=1)[source]

Computes the precision@k for the specified values of k

archai.common.ml_utils.channel_norm(dataset, channel_dim=1) → tuple[source]
archai.common.ml_utils.create_lr_scheduler(conf_lrs: archai.common.config.Config, epochs: int, optimizer: torch.optim.optimizer.Optimizer, steps_per_epoch: Optional[int]) → Tuple[Optional[torch.optim.lr_scheduler._LRScheduler], bool][source]
archai.common.ml_utils.create_optimizer(conf_opt: archai.common.config.Config, params) → torch.optim.optimizer.Optimizer[source]
archai.common.ml_utils.drop_path_(x, drop_prob, training)[source]
archai.common.ml_utils.ensure_pytorch_ver(min_ver: str, error_msg: str) → bool[source]
archai.common.ml_utils.get_lossfn(conf_lossfn: archai.common.config.Config) → torch.nn.modules.loss._Loss[source]
archai.common.ml_utils.get_optim_lr(optimizer: torch.optim.optimizer.Optimizer) → float[source]
archai.common.ml_utils.join_chunks(chunks: List[torch.Tensor]) → torch.Tensor[source]

If batch was divided in chunks, this functions joins them again

archai.common.ml_utils.load_model(model, model_path)[source]
archai.common.ml_utils.param_size(module: torch.nn.modules.module.Module)[source]

count all parameters excluding auxiliary

archai.common.ml_utils.save_model(model, model_path)[source]
archai.common.ml_utils.set_optim_lr(optimizer: torch.optim.optimizer.Optimizer, lr: float) → None[source]

archai.common.model_summary module

archai.common.model_summary.get_all_tensor_stats(o)[source]
archai.common.model_summary.get_shape(o)[source]
archai.common.model_summary.get_tensor_stat(tensor)[source]
archai.common.model_summary.is_scaler(o)[source]
archai.common.model_summary.summary(model, input_size)[source]
archai.common.model_summary.summary_string(model, input_size, dtype=torch.float32)[source]

archai.common.multi_optim module

class archai.common.multi_optim.MultiOptim[source]

Bases: object

append(optim_sched: archai.common.multi_optim.OptimSched) → None[source]
epoch(epoch: Optional[int] = None) → None[source]
get_lr(optim_index: int, param_index: int) → float[source]
load_state_dict(state_dict: dict) → None[source]
state_dict() → dict[source]
step() → None[source]
zero_grad() → None[source]
class archai.common.multi_optim.OptimSched(optim: torch.optim.optimizer.Optimizer, sched: Optional[torch.optim.lr_scheduler._LRScheduler], sched_on_epoch: Optional[bool])[source]

Bases: object

Holds the optimizer and scheduler

archai.common.ordereddict_logger module

class archai.common.ordereddict_logger.OrderedDictLogger(filepath: Optional[str], logger: Optional[logging.Logger], save_delay: Optional[float] = 30.0, yaml_log=True)[source]

Bases: object

The purpose of the structured logging is to store logs as key value pair. However, when you have loop and sub routine calls, what you need is hierarchical dictionaries where the value for a key could be a dictionary. The idea is that you set one of the nodes in tree as current node and start logging your values. You can then use pushd to create and go to child node and popd to come back to parent. To implement this mechanism we use two main variables: _stack allows us to push each node on stack when pushd is called. The node is OrderedDictionary. As a convinience, we let specify child path in pushd in which case child hierarchy is created and current node will be set to the last node in specified path. When popd is called, we go back to original parent instead of parent of current node. To implement this we use _paths variable which stores subpath when each pushd call was made.

close() → None[source]
debug(dict: Union[Mapping, str], level: Optional[int] = 10, exists_ok=False) → None[source]
info(dict: Union[Mapping, str], level: Optional[int] = 20, exists_ok=False) → None[source]
load(filepath: str) → None[source]
path() → str[source]
popd()[source]
pushd(*keys: Any)archai.common.ordereddict_logger.OrderedDictLogger[source]
reset(filepath: Optional[str], logger: Optional[logging.Logger], save_delay: Optional[float] = 30.0, load_existing_file=False, backup_existing_file=True, yaml_log=True) → None[source]
save(filepath: Optional[str] = None) → None[source]
warn(dict: Union[Mapping, str], level: Optional[int] = 30, exists_ok=False) → None[source]

archai.common.stopwatch module

class archai.common.stopwatch.StopWatch[source]

Bases: object

clear(tag=None)[source]
static get()archai.common.stopwatch.StopWatch[source]
get_elapsed(tag=None)[source]
keys()[source]
pause(tag=None)[source]
static set(instance: archai.common.stopwatch.StopWatch) → None[source]
start(tag=None)[source]
stopwatch: StopWatch = None

archai.common.tester module

class archai.common.tester.Tester(conf_val: archai.common.config.Config, model: torch.nn.modules.module.Module, apex: archai.common.apex_utils.ApexUtils)[source]

Bases: overrides.enforce.EnforceOverrides

get_device()[source]
get_metrics() → Optional[archai.common.metrics.Metrics][source]
load_state_dict(state_dict: dict) → None[source]
state_dict() → dict[source]
test(test_dl: torch.utils.data.dataloader.DataLoader)archai.common.metrics.Metrics[source]

archai.common.timing module

class archai.common.timing.MeasureBlockTime(name: str, no_print=True, disable_gc=False)[source]

Bases: object

cur_elapsed() → float[source]
archai.common.timing.MeasureTime(f_py=None, no_print=True, disable_gc=False, name: Optional[str] = None)[source]
archai.common.timing.add_timing(name: str, elapsed: float, no_print=True) → runstats.fast.Statistics[source]
archai.common.timing.clear_timings() → None[source]
archai.common.timing.get_all_timings() → Dict[str, runstats.fast.Statistics][source]
archai.common.timing.get_last(name: str) → float[source]
archai.common.timing.get_timing(name: str) → runstats.fast.Statistics[source]
archai.common.timing.print_all_timings() → None[source]
archai.common.timing.print_timing(name: str) → None[source]

archai.common.trainer module

class archai.common.trainer.Trainer(conf_train: archai.common.config.Config, model: torch.nn.modules.module.Module, checkpoint: Optional[archai.common.checkpoint.CheckPoint])[source]

Bases: overrides.enforce.EnforceOverrides

compute_loss(lossfn: Callable, y: torch.Tensor, logits: torch.Tensor, aux_weight: float, aux_logits: Optional[torch.Tensor]) → torch.Tensor[source]
create_multi_optim(train_len: int)archai.common.multi_optim.MultiOptim[source]
create_optimizer(conf_optim: archai.common.config.Config, params) → torch.optim.optimizer.Optimizer[source]
create_scheduler(conf_sched: archai.common.config.Config, optim: torch.optim.optimizer.Optimizer, steps_per_epoch: int) → Tuple[Optional[torch.optim.lr_scheduler._LRScheduler], bool][source]
epoch() → int[source]
fit(train_dl: torch.utils.data.dataloader.DataLoader, val_dl: Optional[torch.utils.data.dataloader.DataLoader])archai.common.metrics.Metrics[source]
get_device()[source]
get_metrics()archai.common.metrics.Metrics[source]
get_optimizer(index=0) → torch.optim.optimizer.Optimizer[source]
get_scheduler(index=0) → Optional[torch.optim.lr_scheduler._LRScheduler][source]
post_epoch(train_dl: torch.utils.data.dataloader.DataLoader, val_dl: Optional[torch.utils.data.dataloader.DataLoader]) → None[source]
post_fit(train_dl: torch.utils.data.dataloader.DataLoader, val_dl: Optional[torch.utils.data.dataloader.DataLoader]) → None[source]
post_step(x: torch.Tensor, y: torch.Tensor, logits: torch.Tensor, loss: torch.Tensor, steps: int) → None[source]
pre_epoch(train_dl: torch.utils.data.dataloader.DataLoader, val_dl: Optional[torch.utils.data.dataloader.DataLoader]) → None[source]
pre_fit(train_dl: torch.utils.data.dataloader.DataLoader, val_dl: Optional[torch.utils.data.dataloader.DataLoader]) → None[source]
pre_step(x: torch.Tensor, y: torch.Tensor) → None[source]
restore_checkpoint() → None[source]
update_checkpoint(checkpoint: archai.common.checkpoint.CheckPoint) → None[source]

archai.common.utils module

class archai.common.utils.AverageMeter[source]

Bases: object

reset()[source]
update(val, n=1)[source]
archai.common.utils.append_csv_file(filepath: str, new_row: List[Tuple[str, Any]], delimiter='\t')[source]
archai.common.utils.append_to_filename(filepath: str, name_suffix: str, new_ext: Optional[str] = None) → str[source]

Returns ‘/a/b/c/h.f’ for filepath=’/a/b/c/d.e.f’, new_name=’h’

archai.common.utils.change_filepath_ext(filepath: str, new_ext: str) → str[source]

Returns ‘/a/b/c/d.e.g’ for filepath=’/a/b/c/d.e.f’, new_ext=’.g’

archai.common.utils.change_filepath_name(filepath: str, new_name: str, new_ext: Optional[str] = None) → str[source]

Returns ‘/a/b/c/h.f’ for filepath=’/a/b/c/d.e.f’, new_name=’h’

archai.common.utils.copy_dir(src_dir: str, dest_dir: str, use_shutil: bool = True) → None[source]
archai.common.utils.copy_file(src_file: str, dest_dir_or_file: str, preserve_metadata=False, use_shutil: bool = True) → str[source]
archai.common.utils.copy_file_basic(src_file: str, dest_dir_or_file: str) → str[source]
archai.common.utils.create_logger(filepath: Optional[str] = None, name: Optional[str] = None, level=20, enable_stdout=True) → logging.Logger[source]
archai.common.utils.cuda_device_names() → str[source]
archai.common.utils.deep_comp(o1: Any, o2: Any) → bool[source]
archai.common.utils.deep_update(d: MutableMapping, u: Mapping, map_type: Type[MutableMapping] = <class 'dict'>) → MutableMapping[source]
archai.common.utils.dir_downloads() → str[source]
archai.common.utils.download_and_extract_tar(url, download_root, extract_root=None, filename=None, md5=None, **kwargs)[source]
archai.common.utils.exec_shell_command(command: str, print_command=True) → None[source]
archai.common.utils.extract_tar(src, dest=None, gzip=None, delete=False)[source]
archai.common.utils.filepath_ext(filepath: str) → str[source]

Returns ‘d.e.f’ for ‘/a/b/c/d.e.f’

archai.common.utils.filepath_name_ext(filepath: str) → str[source]

Returns ‘.f’ for ‘/a/b/c/d.e.f’

archai.common.utils.filepath_name_only(filepath: str) → str[source]

Returns ‘d.e’ for ‘/a/b/c/d.e.f’

archai.common.utils.filepath_without_ext(filepath: str) → str[source]

Returns ‘/a/b/c/d.e’ for ‘/a/b/c/d.e.f’

archai.common.utils.first_or_default(it: Iterable, default=None)[source]
archai.common.utils.fmt(val: Any) → str[source]
archai.common.utils.full_path(path: str, create=False) → str[source]
archai.common.utils.has_method(o, name)[source]
archai.common.utils.is_debugging() → bool[source]
archai.common.utils.is_main_process() → bool[source]

Returns True if this process was started as main process instead of child process during multiprocessing

archai.common.utils.load_state_dict(val: Any, state_dict: Mapping) → None[source]
archai.common.utils.main_process_pid() → int[source]
archai.common.utils.process_name() → str[source]
archai.common.utils.read_string(filepath: str) → str[source]
archai.common.utils.setup_cuda(seed: Union[float, int], local_rank: int)[source]
archai.common.utils.state_dict(val) → Mapping[source]
archai.common.utils.write_string(filepath: str, content: str) → None[source]
archai.common.utils.zero_file(filepath) → None[source]

Creates or truncates existing file

archai.common.utils.zip_eq(*iterables)[source]

archai.common.warmup_scheduler module

class archai.common.warmup_scheduler.GradualWarmupScheduler(optimizer, multiplier, total_epoch, after_scheduler=None)[source]

Bases: torch.optim.lr_scheduler._LRScheduler

Gradually warm-up(increasing) learning rate in optimizer. Proposed in ‘Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour’.

Args:

optimizer (Optimizer): Wrapped optimizer. multiplier: target learning rate = base lr * multiplier if multiplier > 1.0. if multiplier = 1.0, lr starts from 0 and ends up with the base_lr. total_epoch: target learning rate is reached at total_epoch, gradually after_scheduler: after target_epoch, use this scheduler(eg. ReduceLROnPlateau)

get_lr()[source]
step(epoch=None, metrics=None)[source]
step_ReduceLROnPlateau(metrics, epoch=None)[source]

archai.common.yaml_utils module

archai.common.yaml_utils.is_proper_path(path: str) → bool[source]
archai.common.yaml_utils.resolve_all(root_d: MutableMapping)[source]

Module contents