FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881073         	|  1859.8         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  91.365         	|  20             	|  1827.3         	|  98.252         	|
|  run_training_batch                                                                                                                                                                                         	|  0.084702       	|  20000          	|  1694.0         	|  91.087         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.084335       	|  20000          	|  1686.7         	|  90.692         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.027085       	|  20000          	|  541.7          	|  29.127         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.0047272      	|  20000          	|  94.544         	|  5.0835         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0041128      	|  20000          	|  82.256         	|  4.4228         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  0.67038        	|  38             	|  25.474         	|  1.3697         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  20.736         	|  1              	|  20.736         	|  1.115          	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00048386     	|  20000          	|  9.6773         	|  0.52034        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.23178        	|  20             	|  4.6355         	|  0.24925        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.00013412     	|  20038          	|  2.6874         	|  0.1445         	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  8.6164e-05     	|  20038          	|  1.7266         	|  0.092835       	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  2.7754e-05     	|  20000          	|  0.55507        	|  0.029846       	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.010348       	|  38             	|  0.39324        	|  0.021144       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  1.7053e-05     	|  20000          	|  0.34106        	|  0.018338       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  1.6516e-05     	|  20000          	|  0.33033        	|  0.017761       	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.064777       	|  2              	|  0.12955        	|  0.006966       	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  4.6477e-06     	|  20000          	|  0.092955       	|  0.0049981      	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  3.7687e-06     	|  20000          	|  0.075373       	|  0.0040528      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  3.1411e-06     	|  20000          	|  0.062823       	|  0.0033779      	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  3.1227e-06     	|  20000          	|  0.062455       	|  0.0033581      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  2.9567e-06     	|  20000          	|  0.059133       	|  0.0031795      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  2.8255e-06     	|  20000          	|  0.05651        	|  0.0030385      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  2.6635e-06     	|  20000          	|  0.053271       	|  0.0028643      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  2.6364e-06     	|  20000          	|  0.052727       	|  0.0028351      	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  2.5276e-06     	|  20000          	|  0.050552       	|  0.0027181      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  2.372e-06      	|  20000          	|  0.047439       	|  0.0025508      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  2.3291e-06     	|  20000          	|  0.046582       	|  0.0025047      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  2.2775e-06     	|  20000          	|  0.045551       	|  0.0024492      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.2375e-06     	|  20000          	|  0.04475        	|  0.0024062      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  2.1724e-06     	|  20000          	|  0.043447       	|  0.0023361      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  2.1533e-06     	|  20000          	|  0.043066       	|  0.0023156      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  2.1508e-06     	|  20000          	|  0.043016       	|  0.0023129      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  2.0684e-06     	|  20000          	|  0.041368       	|  0.0022243      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  2.0498e-06     	|  20000          	|  0.040996       	|  0.0022043      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  2.0456e-06     	|  20000          	|  0.040911       	|  0.0021998      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  2.0122e-06     	|  20000          	|  0.040244       	|  0.0021639      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  2.0117e-06     	|  20000          	|  0.040234       	|  0.0021633      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  1.9764e-06     	|  20000          	|  0.039528       	|  0.0021254      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  1.4068e-06     	|  20038          	|  0.02819        	|  0.0015157      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.3146e-06     	|  20000          	|  0.026293       	|  0.0014137      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  1.1518e-06     	|  20000          	|  0.023036       	|  0.0012386      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  1.133e-06      	|  20038          	|  0.022703       	|  0.0012207      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  1.0557e-06     	|  20000          	|  0.021114       	|  0.0011353      	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  1.0473e-06     	|  20000          	|  0.020946       	|  0.0011262      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  9.7807e-07     	|  20000          	|  0.019561       	|  0.0010518      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  9.4904e-07     	|  20000          	|  0.018981       	|  0.0010206      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  8.9606e-07     	|  20000          	|  0.017921       	|  0.00096361     	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  8.6041e-07     	|  20000          	|  0.017208       	|  0.00092527     	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  8.5831e-07     	|  20000          	|  0.017166       	|  0.00092301     	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.0004176      	|  38             	|  0.015869       	|  0.00085325     	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0037247      	|  2              	|  0.0074493      	|  0.00040054     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00027869     	|  20             	|  0.0055738      	|  0.0002997      	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00025319     	|  20             	|  0.0050638      	|  0.00027228     	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.0038844      	|  1              	|  0.0038844      	|  0.00020886     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  7.1606e-05     	|  38             	|  0.002721       	|  0.00014631     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  0.00011631     	|  20             	|  0.0023263      	|  0.00012508     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00052494     	|  2              	|  0.0010499      	|  5.6451e-05     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.00047681     	|  2              	|  0.00095362     	|  5.1275e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00038882     	|  2              	|  0.00077764     	|  4.1813e-05     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00030435     	|  2              	|  0.00060869     	|  3.2729e-05     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00048642     	|  1              	|  0.00048642     	|  2.6155e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.00038769     	|  1              	|  0.00038769     	|  2.0845e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00031818     	|  1              	|  0.00031818     	|  1.7108e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00028938     	|  1              	|  0.00028938     	|  1.556e-05      	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  6.0265e-06     	|  38             	|  0.00022901     	|  1.2314e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  3.8945e-06     	|  38             	|  0.00014799     	|  7.9573e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  6.2599e-06     	|  20             	|  0.0001252      	|  6.7318e-06     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  2.992e-06      	|  38             	|  0.0001137      	|  6.1133e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  2.9765e-06     	|  38             	|  0.00011311     	|  6.0816e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  2.8055e-06     	|  38             	|  0.00010661     	|  5.7322e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  2.7501e-06     	|  38             	|  0.0001045      	|  5.6191e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  5.0901e-06     	|  20             	|  0.0001018      	|  5.4737e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  3.6448e-06     	|  22             	|  8.0185e-05     	|  4.3115e-06     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  2.0671e-06     	|  38             	|  7.8551e-05     	|  4.2236e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  1.8487e-06     	|  38             	|  7.0249e-05     	|  3.7772e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  3.4504e-05     	|  2              	|  6.9008e-05     	|  3.7105e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  3.0837e-06     	|  22             	|  6.784e-05      	|  3.6477e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  3.0814e-06     	|  20             	|  6.1628e-05     	|  3.3137e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  2.2555e-06     	|  22             	|  4.9621e-05     	|  2.6681e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  2.2277e-06     	|  22             	|  4.9009e-05     	|  2.6352e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.2041e-06     	|  22             	|  4.8489e-05     	|  2.6072e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.2001e-06     	|  22             	|  4.8402e-05     	|  2.6025e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  3.686e-05      	|  1              	|  3.686e-05      	|  1.9819e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  1.4018e-06     	|  22             	|  3.0839e-05     	|  1.6582e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.5394e-06     	|  20             	|  3.0788e-05     	|  1.6554e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.3913e-06     	|  22             	|  3.0608e-05     	|  1.6458e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.4656e-06     	|  20             	|  2.9312e-05     	|  1.5761e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  7.8003e-06     	|  2              	|  1.5601e-05     	|  8.3883e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  6.8098e-06     	|  2              	|  1.362e-05      	|  7.3232e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  4.8252e-06     	|  2              	|  9.6504e-06     	|  5.1889e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  8.8606e-06     	|  1              	|  8.8606e-06     	|  4.7643e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  8.5803e-06     	|  1              	|  8.5803e-06     	|  4.6135e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  4.0298e-06     	|  2              	|  8.0597e-06     	|  4.3336e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  3.7542e-06     	|  2              	|  7.5083e-06     	|  4.0372e-07     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  7.0799e-06     	|  1              	|  7.0799e-06     	|  3.8068e-07     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  6.3302e-06     	|  1              	|  6.3302e-06     	|  3.4037e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  3.0999e-06     	|  2              	|  6.1998e-06     	|  3.3336e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  3.0799e-06     	|  2              	|  6.1598e-06     	|  3.312e-07      	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  5.8003e-06     	|  1              	|  5.8003e-06     	|  3.1188e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  2.8247e-06     	|  2              	|  5.6494e-06     	|  3.0376e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  5.2201e-06     	|  1              	|  5.2201e-06     	|  2.8068e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  4.4899e-06     	|  1              	|  4.4899e-06     	|  2.4142e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  2.0848e-06     	|  2              	|  4.1695e-06     	|  2.2419e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  2.0652e-06     	|  2              	|  4.1304e-06     	|  2.2209e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  4.0801e-06     	|  1              	|  4.0801e-06     	|  2.1938e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  2e-06          	|  2              	|  4e-06          	|  2.1508e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  1.9749e-06     	|  2              	|  3.9497e-06     	|  2.1237e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  3.7104e-06     	|  1              	|  3.7104e-06     	|  1.995e-07      	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  1.8352e-06     	|  2              	|  3.6703e-06     	|  1.9735e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  3.6396e-06     	|  1              	|  3.6396e-06     	|  1.957e-07      	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  3.6005e-06     	|  1              	|  3.6005e-06     	|  1.936e-07      	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  3.58e-06       	|  1              	|  3.58e-06       	|  1.9249e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  3.1702e-06     	|  1              	|  3.1702e-06     	|  1.7046e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  3.0799e-06     	|  1              	|  3.0799e-06     	|  1.656e-07      	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  2.9001e-06     	|  1              	|  2.9001e-06     	|  1.5594e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  2.8498e-06     	|  1              	|  2.8498e-06     	|  1.5323e-07     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  1.4054e-06     	|  2              	|  2.8107e-06     	|  1.5113e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  2.7502e-06     	|  1              	|  2.7502e-06     	|  1.4788e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  2.6999e-06     	|  1              	|  2.6999e-06     	|  1.4517e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  2.5705e-06     	|  1              	|  2.5705e-06     	|  1.3821e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  2.4997e-06     	|  1              	|  2.4997e-06     	|  1.344e-07      	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  2.4792e-06     	|  1              	|  2.4792e-06     	|  1.333e-07      	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  2.4689e-06     	|  1              	|  2.4689e-06     	|  1.3275e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  2.4196e-06     	|  1              	|  2.4196e-06     	|  1.301e-07      	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  2.4103e-06     	|  1              	|  2.4103e-06     	|  1.296e-07      	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  2.3805e-06     	|  1              	|  2.3805e-06     	|  1.28e-07       	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  2.36e-06       	|  1              	|  2.36e-06       	|  1.2689e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  2.2603e-06     	|  1              	|  2.2603e-06     	|  1.2154e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  2.1905e-06     	|  1              	|  2.1905e-06     	|  1.1778e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  2.1197e-06     	|  1              	|  2.1197e-06     	|  1.1397e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  2.0899e-06     	|  1              	|  2.0899e-06     	|  1.1237e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  2.0703e-06     	|  1              	|  2.0703e-06     	|  1.1132e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  1.8794e-06     	|  1              	|  1.8794e-06     	|  1.0105e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  1.8599e-06     	|  1              	|  1.8599e-06     	|  1e-07          	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  1.7704e-06     	|  1              	|  1.7704e-06     	|  9.5195e-08     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  1.7602e-06     	|  1              	|  1.7602e-06     	|  9.4644e-08     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  1.6894e-06     	|  1              	|  1.6894e-06     	|  9.0838e-08     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  1.6401e-06     	|  1              	|  1.6401e-06     	|  8.8184e-08     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  1.5702e-06     	|  1              	|  1.5702e-06     	|  8.4429e-08     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  1.099e-06      	|  1              	|  1.099e-06      	|  5.909e-08      	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  8.7917e-07     	|  1              	|  8.7917e-07     	|  4.7272e-08     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
