FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881265         	|  702.27         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  33.714         	|  20             	|  674.28         	|  96.015         	|
|  run_training_batch                                                                                                                                                                                         	|  0.027413       	|  20000          	|  548.26         	|  78.07          	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.027136       	|  20000          	|  542.72         	|  77.281         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.010104       	|  20000          	|  202.07         	|  28.774         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  1.0622         	|  50             	|  53.111         	|  7.5627         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0023728      	|  20000          	|  47.456         	|  6.7576         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.0022778      	|  20000          	|  45.557         	|  6.4871         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  14.92          	|  1              	|  14.92          	|  2.1246         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00048041     	|  20000          	|  9.6083         	|  1.3682         	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.15658        	|  20             	|  3.1316         	|  0.44593        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.00011949     	|  20050          	|  2.3958         	|  0.34115        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  7.6102e-05     	|  20050          	|  1.5258         	|  0.21727        	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.017625       	|  50             	|  0.88125        	|  0.12549        	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  1.8657e-05     	|  20000          	|  0.37314        	|  0.053134       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  1.5472e-05     	|  20000          	|  0.30944        	|  0.044063       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  1.3191e-05     	|  20000          	|  0.26381        	|  0.037566       	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  4.7172e-06     	|  20000          	|  0.094344       	|  0.013434       	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.038908       	|  2              	|  0.077816       	|  0.011081       	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  3.7433e-06     	|  20000          	|  0.074865       	|  0.01066        	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  3.0785e-06     	|  20000          	|  0.06157        	|  0.0087673      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  2.9794e-06     	|  20000          	|  0.059588       	|  0.0084851      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  2.7314e-06     	|  20000          	|  0.054629       	|  0.0077789      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  2.7128e-06     	|  20000          	|  0.054257       	|  0.0077259      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  2.6026e-06     	|  20000          	|  0.052053       	|  0.0074121      	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  2.5825e-06     	|  20000          	|  0.05165        	|  0.0073547      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  2.4531e-06     	|  20000          	|  0.049062       	|  0.0069862      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  2.4261e-06     	|  20000          	|  0.048522       	|  0.0069093      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  2.3963e-06     	|  20000          	|  0.047927       	|  0.0068246      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  2.3762e-06     	|  20000          	|  0.047524       	|  0.0067672      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  2.2208e-06     	|  20000          	|  0.044415       	|  0.0063245      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.1238e-06     	|  20000          	|  0.042476       	|  0.0060484      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  2.099e-06      	|  20000          	|  0.041981       	|  0.0059778      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  2.0975e-06     	|  20000          	|  0.04195        	|  0.0059734      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  2.0943e-06     	|  20000          	|  0.041887       	|  0.0059645      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  2.094e-06      	|  20000          	|  0.04188        	|  0.0059635      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  2.0332e-06     	|  20000          	|  0.040665       	|  0.0057905      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  1.9838e-06     	|  20000          	|  0.039676       	|  0.0056497      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  1.9666e-06     	|  20000          	|  0.039333       	|  0.0056008      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  1.9523e-06     	|  20000          	|  0.039045       	|  0.0055598      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  1.2122e-06     	|  20050          	|  0.024305       	|  0.0034609      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  1.1151e-06     	|  20050          	|  0.022358       	|  0.0031837      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.1014e-06     	|  20000          	|  0.022029       	|  0.0031368      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  1.0912e-06     	|  20000          	|  0.021825       	|  0.0031077      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.00041106     	|  50             	|  0.020553       	|  0.0029266      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  9.6411e-07     	|  20000          	|  0.019282       	|  0.0027457      	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  9.6052e-07     	|  20000          	|  0.01921        	|  0.0027355      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  8.631e-07      	|  20000          	|  0.017262       	|  0.002458       	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  8.3541e-07     	|  20000          	|  0.016708       	|  0.0023792      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  8.2782e-07     	|  20000          	|  0.016556       	|  0.0023575      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  7.7467e-07     	|  20000          	|  0.015493       	|  0.0022062      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  7.6753e-07     	|  20000          	|  0.015351       	|  0.0021859      	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0058188      	|  2              	|  0.011638       	|  0.0016571      	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.010722       	|  1              	|  0.010722       	|  0.0015268      	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00022732     	|  20             	|  0.0045463      	|  0.00064738     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00022076     	|  20             	|  0.0044151      	|  0.0006287      	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  6.7235e-05     	|  50             	|  0.0033618      	|  0.0004787      	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  0.00013964     	|  20             	|  0.0027927      	|  0.00039767     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00085949     	|  2              	|  0.001719       	|  0.00024478     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00044933     	|  2              	|  0.00089865     	|  0.00012796     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.00042494     	|  2              	|  0.00084988     	|  0.00012102     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00037291     	|  2              	|  0.00074581     	|  0.0001062      	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00057004     	|  1              	|  0.00057004     	|  8.1172e-05     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00056073     	|  1              	|  0.00056073     	|  7.9846e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.00032541     	|  1              	|  0.00032541     	|  4.6336e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00030735     	|  1              	|  0.00030735     	|  4.3765e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  5.5557e-06     	|  50             	|  0.00027778     	|  3.9555e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  3.5501e-06     	|  50             	|  0.00017751     	|  2.5276e-05     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  3.1533e-06     	|  50             	|  0.00015767     	|  2.2451e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  2.7807e-06     	|  50             	|  0.00013904     	|  1.9798e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  2.711e-06      	|  50             	|  0.00013555     	|  1.9302e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  6.0876e-06     	|  20             	|  0.00012175     	|  1.7337e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  4.4032e-06     	|  20             	|  8.8065e-05     	|  1.254e-05      	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  1.6547e-06     	|  50             	|  8.2737e-05     	|  1.1781e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  1.545e-06      	|  50             	|  7.7249e-05     	|  1.1e-05        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  3.488e-05      	|  2              	|  6.976e-05      	|  9.9335e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  2.9969e-06     	|  22             	|  6.5931e-05     	|  9.3883e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  2.9759e-06     	|  22             	|  6.547e-05      	|  9.3226e-06     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  1.0952e-06     	|  50             	|  5.4758e-05     	|  7.7973e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  2.7344e-06     	|  20             	|  5.4688e-05     	|  7.7873e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.2264e-06     	|  22             	|  4.8981e-05     	|  6.9747e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.1446e-06     	|  22             	|  4.7181e-05     	|  6.7183e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  2.0292e-06     	|  22             	|  4.4642e-05     	|  6.3568e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  1.9278e-06     	|  22             	|  4.2411e-05     	|  6.0391e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  4.0889e-05     	|  1              	|  4.0889e-05     	|  5.8224e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.5377e-06     	|  20             	|  3.0753e-05     	|  4.3791e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  1.205e-06      	|  22             	|  2.6509e-05     	|  3.7748e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.1614e-06     	|  22             	|  2.5552e-05     	|  3.6385e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.244e-06      	|  20             	|  2.4879e-05     	|  3.5427e-06     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  1.715e-05      	|  1              	|  1.715e-05      	|  2.4421e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  6.3595e-06     	|  2              	|  1.2719e-05     	|  1.8111e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  5.8445e-06     	|  2              	|  1.1689e-05     	|  1.6645e-06     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  1.1669e-05     	|  1              	|  1.1669e-05     	|  1.6617e-06     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  9.7901e-06     	|  1              	|  9.7901e-06     	|  1.3941e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  4.5896e-06     	|  2              	|  9.1791e-06     	|  1.3071e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  3.9698e-06     	|  2              	|  7.9395e-06     	|  1.1306e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  3.8198e-06     	|  2              	|  7.6396e-06     	|  1.0878e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  3.3444e-06     	|  2              	|  6.6888e-06     	|  9.5245e-07     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  6.0303e-06     	|  1              	|  6.0303e-06     	|  8.5869e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  2.9402e-06     	|  2              	|  5.8804e-06     	|  8.3734e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  2.8447e-06     	|  2              	|  5.6894e-06     	|  8.1015e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  5.3002e-06     	|  1              	|  5.3002e-06     	|  7.5472e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  2.59e-06       	|  2              	|  5.18e-06       	|  7.3761e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  2.5798e-06     	|  2              	|  5.1595e-06     	|  7.3469e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  5.0301e-06     	|  1              	|  5.0301e-06     	|  7.1626e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  4.6603e-06     	|  1              	|  4.6603e-06     	|  6.6361e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  4.5402e-06     	|  1              	|  4.5402e-06     	|  6.465e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  4.1798e-06     	|  1              	|  4.1798e-06     	|  5.9518e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  4.0699e-06     	|  1              	|  4.0699e-06     	|  5.7953e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  1.9791e-06     	|  2              	|  3.9581e-06     	|  5.6362e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  3.8901e-06     	|  1              	|  3.8901e-06     	|  5.5394e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  1.7951e-06     	|  2              	|  3.5902e-06     	|  5.1123e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  3.5409e-06     	|  1              	|  3.5409e-06     	|  5.0421e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  3.2308e-06     	|  1              	|  3.2308e-06     	|  4.6005e-07     	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  2.9802e-06     	|  1              	|  2.9802e-06     	|  4.2437e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  2.97e-06       	|  1              	|  2.97e-06       	|  4.2291e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  2.8396e-06     	|  1              	|  2.8396e-06     	|  4.0435e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  1.3453e-06     	|  2              	|  2.6906e-06     	|  3.8313e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  2.5695e-06     	|  1              	|  2.5695e-06     	|  3.6589e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  2.4e-06        	|  1              	|  2.4e-06        	|  3.4175e-07     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  1.1604e-06     	|  2              	|  2.3209e-06     	|  3.3048e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  2.3106e-06     	|  1              	|  2.3106e-06     	|  3.2902e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  2.2901e-06     	|  1              	|  2.2901e-06     	|  3.261e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  2.2696e-06     	|  1              	|  2.2696e-06     	|  3.2319e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  2.2203e-06     	|  1              	|  2.2203e-06     	|  3.1616e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  2.2193e-06     	|  1              	|  2.2193e-06     	|  3.1602e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  2.21e-06       	|  1              	|  2.21e-06       	|  3.147e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  2.21e-06       	|  1              	|  2.21e-06       	|  3.147e-07      	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  2.1895e-06     	|  1              	|  2.1895e-06     	|  3.1178e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  2.1802e-06     	|  1              	|  2.1802e-06     	|  3.1045e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  2.0899e-06     	|  1              	|  2.0899e-06     	|  2.9759e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  2.0694e-06     	|  1              	|  2.0694e-06     	|  2.9467e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  2.0498e-06     	|  1              	|  2.0498e-06     	|  2.9189e-07     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  2.0107e-06     	|  1              	|  2.0107e-06     	|  2.8632e-07     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  2.0098e-06     	|  1              	|  2.0098e-06     	|  2.8619e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  1.9502e-06     	|  1              	|  1.9502e-06     	|  2.777e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  1.9195e-06     	|  1              	|  1.9195e-06     	|  2.7332e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  1.5199e-06     	|  1              	|  1.5199e-06     	|  2.1643e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  1.5004e-06     	|  1              	|  1.5004e-06     	|  2.1364e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  1.3001e-06     	|  1              	|  1.3001e-06     	|  1.8513e-07     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  1.2899e-06     	|  1              	|  1.2899e-06     	|  1.8367e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  1.03e-06       	|  1              	|  1.03e-06       	|  1.4667e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  1.03e-06       	|  1              	|  1.03e-06       	|  1.4667e-07     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
