# archai.algos.divnas package

## Submodules

## archai.algos.divnas.analyse_activations module

`archai.algos.divnas.analyse_activations.``collect_features`(rootfolder: str, subsampling_factor: int = 1) → Dict[str, List[numpy.array]]

  Walks the rootfolder for h5py files and loads them into the format required for analysis.

  Inputs:

  rootfolder: full path to folder containing h5 files which have activations subsampling_factor: every nth minibatch will be loaded to keep memory manageable

  Outputs:

  dictionary with edge name strings as keys and values are lists of np.array [num_samples, feature_dim]

`archai.algos.divnas.analyse_activations.``compute_brute_force_sol`(cov_kernel: numpy.array, budget: int) → Tuple[Tuple[Any], float]

`archai.algos.divnas.analyse_activations.``compute_correlation`(covariance: numpy.array) → numpy.array

`archai.algos.divnas.analyse_activations.``compute_covariance_offline`(feature_list: List[numpy.array]) → numpy.array

  Compute covariance matrix for high-dimensional features. feature_shape: (num_samples, feature_dim)

`archai.algos.divnas.analyse_activations.``compute_euclidean_dist_quantiles`(feature_list: List[numpy.array], subsamplefactor=1) → List[Tuple[float, float]]

  Compute quantile distances between feature pairs feature_list: List of features each of shape: (num_samples, feature_dim)

`archai.algos.divnas.analyse_activations.``compute_marginal_gain`(y: int, A: Set[int], S: Set[int], covariance: numpy.array) → float

`archai.algos.divnas.analyse_activations.``compute_rbf_kernel_covariance`(feature_list: List[numpy.array], sigma=0.1) → numpy.array

  Compute rbf kernel covariance for high dimensional features. feature_list: List of features each of shape: (num_samples, feature_dim) sigma: sigma of the rbf kernel

`archai.algos.divnas.analyse_activations.``create_submod_f`(covariance: numpy.array) → Callable

`archai.algos.divnas.analyse_activations.``get_batch`(feature_list, batch_size, i)

`archai.algos.divnas.analyse_activations.``greedy_op_selection`(covariance: numpy.array, k: int) → List[int]

`archai.algos.divnas.analyse_activations.``main`()

`archai.algos.divnas.analyse_activations.``plot_all_covs`(covs_kernel, corr, primitives, axs)

`archai.algos.divnas.analyse_activations.``rbf`(x: numpy.array, y: numpy.array, sigma=0.1) → numpy.array

  Computes the rbf kernel between two input vectors

## archai.algos.divnas.divnas_cell module

class `archai.algos.divnas.divnas_cell.``Divnas_Cell`(cell: archai.nas.cell.Cell)

  Bases: `object`

  Wrapper cell class for divnas specific modifications

  `clear_collect_activations`()
  `collect_activations`(edgeoptype, sigma: float) → None
  `update_covs`()

`archai.algos.divnas.divnas_cell.``tensor`(data, dtype=None, device=None, requires_grad=False, pin_memory=False) → Tensor

  Constructs a tensor with `data`.

  Warning

  `torch.tensor()` always copies `data`. If you have a Tensor `data` and want to avoid a copy, use `torch.Tensor.requires_grad_()` or `torch.Tensor.detach()`. If you have a NumPy `ndarray` and want to avoid a copy, use `torch.as_tensor()`.

  Warning

  When data is a tensor x, `torch.tensor()` reads out ‘the data’ from whatever it is passed, and constructs a leaf variable. Therefore `torch.tensor(x)` is equivalent to `x.clone().detach()` and `torch.tensor(x, requires_grad=True)` is equivalent to `x.clone().detach().requires_grad_(True)`. The equivalents using `clone()` and `detach()` are recommended.

  Args:
    data (array_like): Initial data for the tensor. Can be a list, tuple,

      NumPy `ndarray`, scalar, and other types.

    dtype (`torch.dtype`, optional): the desired data type of returned tensor.

      Default: if `None`, infers data type from `data`.

    device (`torch.device`, optional): the desired device of returned tensor.

      Default: if `None`, uses the current device for the default tensor type (see `torch.set_default_tensor_type()`). `device` will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.

    requires_grad (bool, optional): If autograd should record operations on the

      returned tensor. Default: `False`.

    pin_memory (bool, optional): If set, returned tensor would be allocated in

      the pinned memory. Works only for CPU tensors. Default: `False`.

  Example:

      >>> torch.tensor([[0.1, 1.2], [2.2, 3.1], [4.9, 5.2]])
      tensor([[ 0.1000,  1.2000],
              [ 2.2000,  3.1000],
              [ 4.9000,  5.2000]])

      >>> torch.tensor([0, 1])  # Type inference on data
      tensor([ 0,  1])

      >>> torch.tensor([[0.11111, 0.222222, 0.3333333]],
                       dtype=torch.float64,
                       device=torch.device('cuda:0'))  # creates a torch.cuda.DoubleTensor
      tensor([[ 0.1111,  0.2222,  0.3333]], dtype=torch.float64, device='cuda:0')

      >>> torch.tensor(3.14159)  # Create a scalar (zero-dimensional tensor)
      tensor(3.1416)

      >>> torch.tensor([])  # Create an empty tensor (of size (0,))
      tensor([])

## archai.algos.divnas.divnas_exp_runner module

class `archai.algos.divnas.divnas_exp_runner.``DivnasExperimentRunner`(config_filename: str, base_name: str, clean_expdir=False)

  Bases: `archai.nas.exp_runner.ExperimentRunner`

  `finalizers`() → archai.nas.finalizers.Finalizers
  `model_desc_builder`() → archai.algos.divnas.divnas_model_desc_builder.DivnasModelDescBuilder
  `trainer_class`() → Optional[Type[archai.nas.arch_trainer.ArchTrainer]]

## archai.algos.divnas.divnas_finalizers module

class `archai.algos.divnas.divnas_finalizers.``DivnasFinalizers`

  Bases: `archai.nas.finalizers.Finalizers`

  `finalize_cell`(cell: archai.nas.cell.Cell, cell_index: int, model_desc: archai.nas.model_desc.ModelDesc, *args, **kwargs) → archai.nas.model_desc.CellDesc
  `finalize_model`(model: archai.nas.model.Model, to_cpu=True, restore_device=True) → archai.nas.model_desc.ModelDesc
  `finalize_node`(node: torch.nn.modules.container.ModuleList, node_index: int, node_desc: archai.nas.model_desc.NodeDesc, max_final_edges: int, *args, **kwargs) → archai.nas.model_desc.NodeDesc

## archai.algos.divnas.divnas_model_desc_builder module

class `archai.algos.divnas.divnas_model_desc_builder.``DivnasModelDescBuilder`

  Bases: `archai.nas.model_desc_builder.ModelDescBuilder`

  `build_nodes`(stem_shapes: List[List[int]], conf_cell: archai.common.config.Config, cell_index: int, cell_type: archai.nas.model_desc.CellType, node_count: int, in_shape: List[int], out_shape: List[int]) → Tuple[List[List[int]], List[archai.nas.model_desc.NodeDesc]]
  `pre_build`(conf_model_desc: archai.common.config.Config) → None

## archai.algos.divnas.divnas_rank_finalizer module

class `archai.algos.divnas.divnas_rank_finalizer.``DivnasRankFinalizers`

  Bases: `archai.nas.finalizers.Finalizers`

  `finalize_cell`(cell: archai.nas.cell.Cell, cell_index: int, model_desc: archai.nas.model_desc.ModelDesc, *args, **kwargs) → archai.nas.model_desc.CellDesc
  `finalize_model`(model: archai.nas.model.Model, to_cpu=True, restore_device=True) → archai.nas.model_desc.ModelDesc
  `finalize_node`(node: torch.nn.modules.container.ModuleList, node_index: int, node_desc: archai.nas.model_desc.NodeDesc, max_final_edges: int, cov: numpy.array, cell: archai.nas.cell.Cell, node_id: int, *args, **kwargs) → archai.nas.model_desc.NodeDesc

## archai.algos.divnas.divop module

class `archai.algos.divnas.divop.``DivOp`(op_desc: archai.nas.model_desc.OpDesc, arch_params: Optional[archai.nas.arch_params.ArchParams], affine: bool)

  Bases: `archai.nas.operations.Op`

  The output of DivOp is weighted output of all allowed primitives.

  `PRIMITIVES` = ['max_pool_3x3', 'avg_pool_3x3', 'skip_connect', 'sep_conv_3x3', 'sep_conv_5x5', 'dil_conv_3x3', 'dil_conv_5x5', 'none']
  property `activations`
  `can_drop_path`() → bool
  property `collect_activations`
  `finalize`() → Tuple[archai.nas.model_desc.OpDesc, Optional[float]]

    Divnas with default finalizer option needs this override else the finalizer in base class returns the whole divop

  `forward`(x)

    Defines the computation performed at every call.

    Should be overridden by all subclasses.

    Note

    Although the recipe for forward pass needs to be defined within this function, one should call the `Module` instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

  property `num_primitive_ops`
  `ops`() → Iterator[Tuple[archai.nas.operations.Op, float]]

    Return contituent ops, if this op is primitive just return self

## archai.algos.divnas.seqopt module

class `archai.algos.divnas.seqopt.``SeqOpt`(num_items: int, eps: float)

  Bases: `object`

  Implements SeqOpt TODO: Later on we might want to refactor this class to be able to handle bandit feedback

  `sample_sequence`(with_replacement=False) → List[int]
  `update`(sel_list: List[int], compute_marginal_gain_func) → None

    In the full information case we will update all expert copies according to the marginal benefits

## archai.algos.divnas.wmr module

class `archai.algos.divnas.wmr.``Wmr`(num_items: int, eta: float)

  Bases: `object`

  Implements the Randomized Weighted Majority algorithm by Littlestone and Warmuth We use the version in Fig 1 in The Multiplicative Weight Update with the gain version

  `sample`() → int
  `update`(rewards: numpy.array) → None
  property `weights`

## Module contents