FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881393         	|  1264.3         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  60.547         	|  20             	|  1210.9         	|  95.775         	|
|  run_training_batch                                                                                                                                                                                         	|  0.049336       	|  20000          	|  986.73         	|  78.042         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.04895        	|  20000          	|  979.01         	|  77.432         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.01505        	|  20000          	|  301.0          	|  23.807         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0080759      	|  20000          	|  161.52         	|  12.775         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.0022852      	|  20000          	|  45.704         	|  3.6149         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  38.121         	|  1              	|  38.121         	|  3.0151         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  0.47565        	|  58             	|  27.588         	|  2.182          	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00066101     	|  20000          	|  13.22          	|  1.0456         	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.00024204     	|  20058          	|  4.8547         	|  0.38397        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  0.00015625     	|  20058          	|  3.1341         	|  0.24788        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.12812        	|  20             	|  2.5624         	|  0.20266        	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  3.3806e-05     	|  20000          	|  0.67612        	|  0.053476       	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.0096835      	|  58             	|  0.56164        	|  0.044422       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  2.0961e-05     	|  20000          	|  0.41923        	|  0.033158       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  2.0305e-05     	|  20000          	|  0.4061         	|  0.03212        	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.058361       	|  2              	|  0.11672        	|  0.0092318      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  5.2134e-06     	|  20000          	|  0.10427        	|  0.0082467      	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  4.7302e-06     	|  20000          	|  0.094604       	|  0.0074825      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  4.576e-06      	|  20000          	|  0.09152        	|  0.0072385      	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  4.3364e-06     	|  20000          	|  0.086729       	|  0.0068596      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  3.8e-06        	|  20058          	|  0.07622        	|  0.0060284      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  3.608e-06      	|  20000          	|  0.072161       	|  0.0057073      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  3.572e-06      	|  20000          	|  0.07144        	|  0.0056504      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  3.4586e-06     	|  20000          	|  0.069172       	|  0.005471       	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  3.4153e-06     	|  20000          	|  0.068305       	|  0.0054024      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  3.202e-06      	|  20000          	|  0.06404        	|  0.0050651      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  3.1741e-06     	|  20000          	|  0.063482       	|  0.0050209      	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  2.768e-06      	|  20000          	|  0.05536        	|  0.0043785      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  2.7068e-06     	|  20000          	|  0.054136       	|  0.0042818      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  2.6594e-06     	|  20000          	|  0.053188       	|  0.0042067      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  2.6161e-06     	|  20000          	|  0.052322       	|  0.0041383      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.544e-06      	|  20000          	|  0.05088        	|  0.0040242      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  2.486e-06      	|  20000          	|  0.049719       	|  0.0039324      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.0008512      	|  58             	|  0.04937        	|  0.0039048      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  2.4532e-06     	|  20000          	|  0.049064       	|  0.0038806      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  2.3383e-06     	|  20000          	|  0.046766       	|  0.0036989      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  2.3086e-06     	|  20000          	|  0.046172       	|  0.0036518      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  2.308e-06      	|  20000          	|  0.04616        	|  0.0036509      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  2.1901e-06     	|  20000          	|  0.043803       	|  0.0034645      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  2.1729e-06     	|  20000          	|  0.043459       	|  0.0034372      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  1.6837e-06     	|  20058          	|  0.033771       	|  0.0026711      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  1.6113e-06     	|  20000          	|  0.032226       	|  0.0025489      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  1.5168e-06     	|  20000          	|  0.030336       	|  0.0023994      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.5144e-06     	|  20000          	|  0.030289       	|  0.0023956      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  1.4184e-06     	|  20000          	|  0.028368       	|  0.0022437      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  1.3151e-06     	|  20000          	|  0.026302       	|  0.0020803      	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  1.2988e-06     	|  20000          	|  0.025975       	|  0.0020544      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  1.1724e-06     	|  20000          	|  0.023448       	|  0.0018545      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  1.168e-06      	|  20000          	|  0.02336        	|  0.0018476      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  1.0995e-06     	|  20000          	|  0.021989       	|  0.0017392      	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0060559      	|  2              	|  0.012112       	|  0.00095795     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00028739     	|  20             	|  0.0057478      	|  0.00045461     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00025605     	|  20             	|  0.005121       	|  0.00040503     	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.0046276      	|  1              	|  0.0046276      	|  0.000366       	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  4.5959e-05     	|  58             	|  0.0026656      	|  0.00021083     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  0.00011896     	|  20             	|  0.0023792      	|  0.00018818     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.0008226      	|  2              	|  0.0016452      	|  0.00013012     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00057046     	|  2              	|  0.0011409      	|  9.0238e-05     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.00053173     	|  2              	|  0.0010635      	|  8.4111e-05     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00040236     	|  2              	|  0.00080472     	|  6.3647e-05     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.0005158      	|  1              	|  0.0005158      	|  4.0796e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00042319     	|  1              	|  0.00042319     	|  3.3471e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00040283     	|  1              	|  0.00040283     	|  3.186e-05      	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  6.7574e-06     	|  58             	|  0.00039193     	|  3.0999e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.00034212     	|  1              	|  0.00034212     	|  2.7059e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  3.9363e-06     	|  58             	|  0.0002283      	|  1.8057e-05     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  3.7168e-06     	|  58             	|  0.00021557     	|  1.705e-05      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  3.1211e-06     	|  58             	|  0.00018103     	|  1.4318e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  2.8958e-06     	|  58             	|  0.00016796     	|  1.3284e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  6.5185e-06     	|  20             	|  0.00013037     	|  1.0311e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  2.1535e-06     	|  58             	|  0.0001249      	|  9.8787e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  1.8555e-06     	|  58             	|  0.00010762     	|  8.5118e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  5.2981e-06     	|  20             	|  0.00010596     	|  8.3807e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  3.7098e-06     	|  22             	|  8.1615e-05     	|  6.4551e-06     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  1.4054e-06     	|  58             	|  8.1511e-05     	|  6.4469e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  3.8795e-05     	|  2              	|  7.7589e-05     	|  6.1367e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  3.3222e-06     	|  22             	|  7.3087e-05     	|  5.7806e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  3.2844e-06     	|  20             	|  6.5687e-05     	|  5.1953e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  2.5033e-06     	|  22             	|  5.5072e-05     	|  4.3558e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  2.2437e-06     	|  22             	|  4.9361e-05     	|  3.9041e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.2264e-06     	|  22             	|  4.898e-05      	|  3.8739e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.1823e-06     	|  22             	|  4.801e-05      	|  3.7972e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  3.7999e-05     	|  1              	|  3.7999e-05     	|  3.0054e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.8198e-06     	|  20             	|  3.6395e-05     	|  2.8786e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.5049e-06     	|  22             	|  3.3108e-05     	|  2.6186e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  1.4854e-06     	|  22             	|  3.2679e-05     	|  2.5847e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.59e-06       	|  20             	|  3.1799e-05     	|  2.5151e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  7.2252e-06     	|  2              	|  1.445e-05      	|  1.1429e-06     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  6.679e-06      	|  2              	|  1.3358e-05     	|  1.0565e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  5.275e-06      	|  2              	|  1.055e-05      	|  8.3443e-07     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  1.032e-05      	|  1              	|  1.032e-05      	|  8.1623e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  4.5053e-06     	|  2              	|  9.0105e-06     	|  7.1266e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  8.0299e-06     	|  1              	|  8.0299e-06     	|  6.351e-07      	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  7.6201e-06     	|  1              	|  7.6201e-06     	|  6.0269e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  7.39e-06       	|  1              	|  7.39e-06       	|  5.845e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  3.635e-06      	|  2              	|  7.2699e-06     	|  5.7499e-07     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  6.8992e-06     	|  1              	|  6.8992e-06     	|  5.4568e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  3.2648e-06     	|  2              	|  6.5295e-06     	|  5.1643e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  2.8247e-06     	|  2              	|  5.6494e-06     	|  4.4682e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  5.6103e-06     	|  1              	|  5.6103e-06     	|  4.4373e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  5.4603e-06     	|  1              	|  5.4603e-06     	|  4.3187e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  5.3896e-06     	|  1              	|  5.3896e-06     	|  4.2627e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  2.5602e-06     	|  2              	|  5.1204e-06     	|  4.0498e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  4.8e-06        	|  1              	|  4.8e-06        	|  3.7965e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  4.6799e-06     	|  1              	|  4.6799e-06     	|  3.7014e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  2.2403e-06     	|  2              	|  4.4806e-06     	|  3.5438e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  4.3092e-06     	|  1              	|  4.3092e-06     	|  3.4083e-07     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  4.2897e-06     	|  1              	|  4.2897e-06     	|  3.3928e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  4.2003e-06     	|  1              	|  4.2003e-06     	|  3.3221e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  2.0554e-06     	|  2              	|  4.1109e-06     	|  3.2514e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  3.7998e-06     	|  1              	|  3.7998e-06     	|  3.0053e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  3.7896e-06     	|  1              	|  3.7896e-06     	|  2.9972e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  1.865e-06      	|  2              	|  3.7299e-06     	|  2.9501e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  1.8398e-06     	|  2              	|  3.6797e-06     	|  2.9103e-07     	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  3.5204e-06     	|  1              	|  3.5204e-06     	|  2.7844e-07     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  3.0901e-06     	|  1              	|  3.0901e-06     	|  2.4441e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  3.0408e-06     	|  1              	|  3.0408e-06     	|  2.405e-07      	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  3.0193e-06     	|  1              	|  3.0193e-06     	|  2.3881e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  2.9504e-06     	|  1              	|  2.9504e-06     	|  2.3336e-07     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  1.4198e-06     	|  2              	|  2.8396e-06     	|  2.2459e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  2.6608e-06     	|  1              	|  2.6608e-06     	|  2.1045e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  2.6403e-06     	|  1              	|  2.6403e-06     	|  2.0883e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  2.6403e-06     	|  1              	|  2.6403e-06     	|  2.0883e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  1.3057e-06     	|  2              	|  2.6114e-06     	|  2.0654e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  2.5099e-06     	|  1              	|  2.5099e-06     	|  1.9851e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  2.4596e-06     	|  1              	|  2.4596e-06     	|  1.9454e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  2.4401e-06     	|  1              	|  2.4401e-06     	|  1.9299e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  2.4298e-06     	|  1              	|  2.4298e-06     	|  1.9218e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  2.3898e-06     	|  1              	|  2.3898e-06     	|  1.8901e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  2.3199e-06     	|  1              	|  2.3199e-06     	|  1.8349e-07     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  2.2706e-06     	|  1              	|  2.2706e-06     	|  1.7958e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  2.2706e-06     	|  1              	|  2.2706e-06     	|  1.7958e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  2.1998e-06     	|  1              	|  2.1998e-06     	|  1.7399e-07     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  2.1597e-06     	|  1              	|  2.1597e-06     	|  1.7082e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  2.0899e-06     	|  1              	|  2.0899e-06     	|  1.6529e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  2.0601e-06     	|  1              	|  2.0601e-06     	|  1.6294e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  2.0396e-06     	|  1              	|  2.0396e-06     	|  1.6132e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  1.9092e-06     	|  1              	|  1.9092e-06     	|  1.51e-07       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  1.8897e-06     	|  1              	|  1.8897e-06     	|  1.4946e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  1.6401e-06     	|  1              	|  1.6401e-06     	|  1.2972e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  1.4696e-06     	|  1              	|  1.4696e-06     	|  1.1624e-07     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
