FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881489         	|  1102.4         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  53.489         	|  20             	|  1069.8         	|  97.038         	|
|  run_training_batch                                                                                                                                                                                         	|  0.04615        	|  20000          	|  922.99         	|  83.724         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.045887       	|  20000          	|  917.74         	|  83.247         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.012625       	|  20000          	|  252.5          	|  22.904         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0036488      	|  20000          	|  72.976         	|  6.6196         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  0.7869         	|  64             	|  50.362         	|  4.5683         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.0019512      	|  20000          	|  39.025         	|  3.5399         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  16.345         	|  1              	|  16.345         	|  1.4827         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00045618     	|  20000          	|  9.1237         	|  0.8276         	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.27534        	|  20             	|  5.5068         	|  0.49951        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.00012408     	|  20064          	|  2.4896         	|  0.22583        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  7.9033e-05     	|  20064          	|  1.5857         	|  0.14384        	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.0071161      	|  64             	|  0.45543        	|  0.041312       	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  2.0883e-05     	|  20000          	|  0.41766        	|  0.037886       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  1.447e-05      	|  20000          	|  0.2894         	|  0.026252       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  1.4406e-05     	|  20000          	|  0.28811        	|  0.026134       	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.045398       	|  2              	|  0.090796       	|  0.008236       	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  4.5043e-06     	|  20000          	|  0.090085       	|  0.0081715      	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  3.5184e-06     	|  20000          	|  0.070369       	|  0.0063831      	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  3.3223e-06     	|  20000          	|  0.066445       	|  0.0060272      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  3.1172e-06     	|  20000          	|  0.062343       	|  0.0056551      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  3.0051e-06     	|  20000          	|  0.060103       	|  0.0054519      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  2.8732e-06     	|  20000          	|  0.057465       	|  0.0052126      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  2.6708e-06     	|  20000          	|  0.053415       	|  0.0048452      	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  2.5257e-06     	|  20000          	|  0.050513       	|  0.004582       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  2.4126e-06     	|  20000          	|  0.048252       	|  0.0043768      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  2.3714e-06     	|  20000          	|  0.047427       	|  0.0043021      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  2.3571e-06     	|  20000          	|  0.047142       	|  0.0042762      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  2.3198e-06     	|  20000          	|  0.046397       	|  0.0042086      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.2835e-06     	|  20000          	|  0.04567        	|  0.0041427      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  2.2354e-06     	|  20000          	|  0.044709       	|  0.0040555      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  2.2127e-06     	|  20000          	|  0.044255       	|  0.0040143      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  2.1939e-06     	|  20000          	|  0.043878       	|  0.0039801      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  2.1789e-06     	|  20000          	|  0.043578       	|  0.0039529      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  2.1565e-06     	|  20000          	|  0.043129       	|  0.0039122      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  2.1397e-06     	|  20000          	|  0.042795       	|  0.0038818      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  2.1265e-06     	|  20000          	|  0.04253        	|  0.0038578      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  2.0431e-06     	|  20000          	|  0.040861       	|  0.0037065      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  2.0368e-06     	|  20000          	|  0.040735       	|  0.003695       	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.00042246     	|  64             	|  0.027037       	|  0.0024525      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  1.3467e-06     	|  20064          	|  0.027019       	|  0.0024509      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  1.2097e-06     	|  20000          	|  0.024194       	|  0.0021946      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.1696e-06     	|  20000          	|  0.023391       	|  0.0021218      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  1.1264e-06     	|  20064          	|  0.0226         	|  0.00205        	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  1.0725e-06     	|  20000          	|  0.02145        	|  0.0019457      	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  1.034e-06      	|  20000          	|  0.020679       	|  0.0018758      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  1.0172e-06     	|  20000          	|  0.020344       	|  0.0018454      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  9.5352e-07     	|  20000          	|  0.01907        	|  0.0017299      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  9.2698e-07     	|  20000          	|  0.01854        	|  0.0016817      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  8.7163e-07     	|  20000          	|  0.017433       	|  0.0015813      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  8.627e-07      	|  20000          	|  0.017254       	|  0.0015651      	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0046181      	|  2              	|  0.0092363      	|  0.00083781     	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.0060587      	|  1              	|  0.0060587      	|  0.00054958     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00027278     	|  20             	|  0.0054555      	|  0.00049487     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00024093     	|  20             	|  0.0048187      	|  0.0004371      	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  4.1926e-05     	|  64             	|  0.0026833      	|  0.0002434      	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  0.00010097     	|  20             	|  0.0020194      	|  0.00018318     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.00057916     	|  2              	|  0.0011583      	|  0.00010507     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.000548       	|  2              	|  0.001096       	|  9.9417e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00048273     	|  2              	|  0.00096546     	|  8.7576e-05     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00061617     	|  1              	|  0.00061617     	|  5.5892e-05     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00023367     	|  2              	|  0.00046734     	|  4.2391e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  5.9633e-06     	|  64             	|  0.00038165     	|  3.4619e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00032597     	|  1              	|  0.00032597     	|  2.9568e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.00030687     	|  1              	|  0.00030687     	|  2.7836e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00030283     	|  1              	|  0.00030283     	|  2.7469e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  3.5795e-06     	|  64             	|  0.00022909     	|  2.078e-05      	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  3.0013e-06     	|  64             	|  0.00019208     	|  1.7424e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  2.9588e-06     	|  64             	|  0.00018936     	|  1.7177e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  2.8392e-06     	|  64             	|  0.00018171     	|  1.6483e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  1.8292e-06     	|  64             	|  0.00011707     	|  1.0619e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  5.7436e-06     	|  20             	|  0.00011487     	|  1.042e-05      	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  4.7489e-06     	|  20             	|  9.4977e-05     	|  8.6153e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  1.4551e-06     	|  64             	|  9.3127e-05     	|  8.4474e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  3.8868e-06     	|  22             	|  8.5509e-05     	|  7.7565e-06     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  1.2894e-06     	|  64             	|  8.2522e-05     	|  7.4854e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  3.4612e-06     	|  22             	|  7.6147e-05     	|  6.9072e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  3.7515e-05     	|  2              	|  7.5029e-05     	|  6.8058e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  2.7489e-06     	|  20             	|  5.4978e-05     	|  4.987e-06      	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  2.1652e-06     	|  22             	|  4.7633e-05     	|  4.3208e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  2.115e-06      	|  22             	|  4.6531e-05     	|  4.2207e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.1115e-06     	|  22             	|  4.6453e-05     	|  4.2137e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.0555e-06     	|  22             	|  4.5221e-05     	|  4.102e-06      	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.529e-06      	|  20             	|  3.0579e-05     	|  2.7738e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  3.0191e-05     	|  1              	|  3.0191e-05     	|  2.7386e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  1.3287e-06     	|  22             	|  2.923e-05      	|  2.6515e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.2828e-06     	|  22             	|  2.8222e-05     	|  2.56e-06       	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.1904e-06     	|  20             	|  2.3808e-05     	|  2.1596e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  5.6997e-06     	|  2              	|  1.1399e-05     	|  1.034e-06      	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  5.35e-06       	|  2              	|  1.07e-05       	|  9.7058e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  4.4596e-06     	|  2              	|  8.9193e-06     	|  8.0906e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  4.0946e-06     	|  2              	|  8.1891e-06     	|  7.4283e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  3.9847e-06     	|  2              	|  7.9693e-06     	|  7.2289e-07     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  7.3295e-06     	|  1              	|  7.3295e-06     	|  6.6485e-07     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  6.9896e-06     	|  1              	|  6.9896e-06     	|  6.3402e-07     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  6.7092e-06     	|  1              	|  6.7092e-06     	|  6.0859e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  3.2857e-06     	|  2              	|  6.5714e-06     	|  5.9609e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  6.32e-06       	|  1              	|  6.32e-06       	|  5.7328e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  6.1393e-06     	|  1              	|  6.1393e-06     	|  5.5689e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  2.705e-06      	|  2              	|  5.4101e-06     	|  4.9074e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  2.6352e-06     	|  2              	|  5.2704e-06     	|  4.7807e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  2.5849e-06     	|  2              	|  5.1698e-06     	|  4.6894e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  4.95e-06       	|  1              	|  4.95e-06       	|  4.4901e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  4.8e-06        	|  1              	|  4.8e-06        	|  4.3541e-07     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  4.6901e-06     	|  1              	|  4.6901e-06     	|  4.2544e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  2.3204e-06     	|  2              	|  4.6408e-06     	|  4.2096e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  2.285e-06      	|  2              	|  4.57e-06       	|  4.1454e-07     	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  4.5598e-06     	|  1              	|  4.5598e-06     	|  4.1361e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  4.2897e-06     	|  1              	|  4.2897e-06     	|  3.8911e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  4.2403e-06     	|  1              	|  4.2403e-06     	|  3.8463e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  4.2394e-06     	|  1              	|  4.2394e-06     	|  3.8455e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  2.0699e-06     	|  2              	|  4.1397e-06     	|  3.7551e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  2.0447e-06     	|  2              	|  4.0894e-06     	|  3.7095e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  3.6499e-06     	|  1              	|  3.6499e-06     	|  3.3107e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  3.1395e-06     	|  1              	|  3.1395e-06     	|  2.8478e-07     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  2.9905e-06     	|  1              	|  2.9905e-06     	|  2.7126e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  2.7707e-06     	|  1              	|  2.7707e-06     	|  2.5133e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  2.6803e-06     	|  1              	|  2.6803e-06     	|  2.4313e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  2.5593e-06     	|  1              	|  2.5593e-06     	|  2.3215e-07     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  1.275e-06      	|  2              	|  2.55e-06       	|  2.313e-07      	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  2.55e-06       	|  1              	|  2.55e-06       	|  2.313e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  2.5397e-06     	|  1              	|  2.5397e-06     	|  2.3037e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  2.4997e-06     	|  1              	|  2.4997e-06     	|  2.2674e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  2.4503e-06     	|  1              	|  2.4503e-06     	|  2.2226e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  2.4196e-06     	|  1              	|  2.4196e-06     	|  2.1948e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  2.3805e-06     	|  1              	|  2.3805e-06     	|  2.1593e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  2.2594e-06     	|  1              	|  2.2594e-06     	|  2.0495e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  2.1998e-06     	|  1              	|  2.1998e-06     	|  1.9954e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  2.17e-06       	|  1              	|  2.17e-06       	|  1.9684e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  2.17e-06       	|  1              	|  2.17e-06       	|  1.9684e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  2.1402e-06     	|  1              	|  2.1402e-06     	|  1.9413e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  2.0899e-06     	|  1              	|  2.0899e-06     	|  1.8957e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  2.0405e-06     	|  1              	|  2.0405e-06     	|  1.8509e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  1.9697e-06     	|  1              	|  1.9697e-06     	|  1.7867e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  1.9697e-06     	|  1              	|  1.9697e-06     	|  1.7867e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  1.9195e-06     	|  1              	|  1.9195e-06     	|  1.7411e-07     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  1.8198e-06     	|  1              	|  1.8198e-06     	|  1.6507e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  1.6605e-06     	|  1              	|  1.6605e-06     	|  1.5063e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  1.6298e-06     	|  1              	|  1.6298e-06     	|  1.4784e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  1.3802e-06     	|  1              	|  1.3802e-06     	|  1.252e-07      	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  1.3197e-06     	|  1              	|  1.3197e-06     	|  1.1971e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  1.0403e-06     	|  1              	|  1.0403e-06     	|  9.4363e-08     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
