This study has several limitations. Firstly, calculating the FIM in weight space rather than output space makes it intractable to compute the FIM for the entire model parameter. Obtaining the FIM for the entire model weight space could lead to much more powerful performance improvements, making it one potential direction for future work. Secondly, the assumption of the existence of pre-trained models is necessary for Bayesian transfer learning. In situations where pre-trained weights are not available, performance improvements may be somewhat limited.
