FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881265         	|  1508.6         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  73.989         	|  20             	|  1479.8         	|  98.088         	|
|  run_training_batch                                                                                                                                                                                         	|  0.061784       	|  20000          	|  1235.7         	|  81.907         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.061357       	|  20000          	|  1227.1         	|  81.341         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.02234        	|  20000          	|  446.8          	|  29.617         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0068312      	|  20000          	|  136.62         	|  9.0562         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.0048222      	|  20000          	|  96.444         	|  6.3928         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  1.5429         	|  50             	|  77.143         	|  5.1134         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  16.394         	|  1              	|  16.394         	|  1.0867         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00054888     	|  20000          	|  10.978         	|  0.72766        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.23725        	|  20             	|  4.745          	|  0.31453        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.00018118     	|  20050          	|  3.6326         	|  0.24079        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  0.00011915     	|  20050          	|  2.3889         	|  0.15835        	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  3.6521e-05     	|  20000          	|  0.73043        	|  0.048417       	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.012614       	|  50             	|  0.63071        	|  0.041807       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  2.2053e-05     	|  20000          	|  0.44105        	|  0.029235       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  2.0919e-05     	|  20000          	|  0.41838        	|  0.027732       	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  5.6247e-06     	|  20000          	|  0.11249        	|  0.0074567      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  5.0845e-06     	|  20000          	|  0.10169        	|  0.0067406      	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.050739       	|  2              	|  0.10148        	|  0.0067265      	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  4.308e-06      	|  20000          	|  0.08616        	|  0.0057111      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  3.6699e-06     	|  20000          	|  0.073399       	|  0.0048653      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  3.2874e-06     	|  20000          	|  0.065748       	|  0.0043581      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  3.236e-06      	|  20000          	|  0.06472        	|  0.00429        	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  3.2297e-06     	|  20000          	|  0.064593       	|  0.0042816      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  2.8388e-06     	|  20000          	|  0.056777       	|  0.0037635      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.811e-06      	|  20000          	|  0.05622        	|  0.0037266      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  2.7241e-06     	|  20000          	|  0.054483       	|  0.0036114      	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  2.6978e-06     	|  20000          	|  0.053955       	|  0.0035765      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  2.6739e-06     	|  20000          	|  0.053477       	|  0.0035448      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  2.4734e-06     	|  20000          	|  0.049469       	|  0.0032791      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  2.4621e-06     	|  20000          	|  0.049242       	|  0.003264       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  2.3714e-06     	|  20000          	|  0.047428       	|  0.0031438      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  2.3361e-06     	|  20000          	|  0.046721       	|  0.0030969      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  2.3326e-06     	|  20000          	|  0.046652       	|  0.0030924      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  2.3233e-06     	|  20000          	|  0.046466       	|  0.00308        	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  2.2084e-06     	|  20000          	|  0.044167       	|  0.0029277      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  2.1604e-06     	|  20000          	|  0.043209       	|  0.0028641      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  2.0914e-06     	|  20000          	|  0.041829       	|  0.0027726      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  2.03e-06       	|  20000          	|  0.040601       	|  0.0026913      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.0007113      	|  50             	|  0.035565       	|  0.0023574      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  1.7488e-06     	|  20050          	|  0.035064       	|  0.0023243      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  1.7254e-06     	|  20050          	|  0.034594       	|  0.002293       	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.358e-06      	|  20000          	|  0.027161       	|  0.0018004      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  1.3005e-06     	|  20000          	|  0.02601        	|  0.0017241      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  1.2461e-06     	|  20000          	|  0.024922       	|  0.001652       	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  1.2124e-06     	|  20000          	|  0.024248       	|  0.0016073      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  1.1949e-06     	|  20000          	|  0.023898       	|  0.0015841      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  1.1821e-06     	|  20000          	|  0.023641       	|  0.0015671      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  1.1678e-06     	|  20000          	|  0.023356       	|  0.0015482      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  1.1177e-06     	|  20000          	|  0.022354       	|  0.0014817      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  1.0452e-06     	|  20000          	|  0.020903       	|  0.0013856      	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00049708     	|  20             	|  0.0099416      	|  0.00065899     	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0046823      	|  2              	|  0.0093646      	|  0.00062074     	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.0070017      	|  1              	|  0.0070017      	|  0.00046411     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00025898     	|  20             	|  0.0051796      	|  0.00034333     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  9.9694e-05     	|  20             	|  0.0019939      	|  0.00013217     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  3.4602e-05     	|  50             	|  0.0017301      	|  0.00011468     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.00051887     	|  2              	|  0.0010377      	|  6.8786e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00048266     	|  2              	|  0.00096532     	|  6.3987e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00040099     	|  2              	|  0.00080198     	|  5.316e-05      	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00073755     	|  1              	|  0.00073755     	|  4.8889e-05     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.0003281      	|  2              	|  0.00065619     	|  4.3496e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.00042602     	|  1              	|  0.00042602     	|  2.8239e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  6.8965e-06     	|  50             	|  0.00034483     	|  2.2857e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00030716     	|  1              	|  0.00030716     	|  2.036e-05      	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00029218     	|  1              	|  0.00029218     	|  1.9367e-05     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  3.8228e-06     	|  50             	|  0.00019114     	|  1.267e-05      	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  8.8699e-06     	|  20             	|  0.0001774      	|  1.1759e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  3.3564e-06     	|  50             	|  0.00016782     	|  1.1124e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  2.9416e-06     	|  50             	|  0.00014708     	|  9.7492e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  2.5899e-06     	|  50             	|  0.0001295      	|  8.5837e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  2.356e-06      	|  50             	|  0.0001178      	|  7.8084e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  4.9908e-06     	|  20             	|  9.9816e-05     	|  6.6164e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  3.56e-06       	|  22             	|  7.832e-05      	|  5.1914e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  1.482e-06      	|  50             	|  7.4101e-05     	|  4.9118e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  3.5495e-05     	|  2              	|  7.099e-05      	|  4.7056e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  3.1053e-06     	|  22             	|  6.8316e-05     	|  4.5284e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  3.1891e-06     	|  20             	|  6.3782e-05     	|  4.2278e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  2.7267e-06     	|  22             	|  5.9988e-05     	|  3.9764e-06     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  1.1808e-06     	|  50             	|  5.904e-05      	|  3.9135e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  5.208e-05      	|  1              	|  5.208e-05      	|  3.4521e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  2.3271e-06     	|  22             	|  5.1197e-05     	|  3.3936e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.2255e-06     	|  22             	|  4.8961e-05     	|  3.2454e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.2172e-06     	|  22             	|  4.8778e-05     	|  3.2333e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  1.6031e-06     	|  22             	|  3.5268e-05     	|  2.3378e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.6142e-06     	|  20             	|  3.2283e-05     	|  2.1399e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.3927e-06     	|  22             	|  3.0639e-05     	|  2.0309e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.4685e-06     	|  20             	|  2.937e-05      	|  1.9468e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  1.052e-05      	|  2              	|  2.104e-05      	|  1.3946e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  6.59e-06       	|  2              	|  1.318e-05      	|  8.7365e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  5.8701e-06     	|  2              	|  1.174e-05      	|  7.7821e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  5.4999e-06     	|  2              	|  1.1e-05        	|  7.2913e-07     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  1.05e-05       	|  1              	|  1.05e-05       	|  6.9598e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  4.7847e-06     	|  2              	|  9.5693e-06     	|  6.3431e-07     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  8.6101e-06     	|  1              	|  8.6101e-06     	|  5.7072e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  3.9395e-06     	|  2              	|  7.879e-06      	|  5.2226e-07     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  7.5502e-06     	|  1              	|  7.5502e-06     	|  5.0047e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  3.315e-06      	|  2              	|  6.6301e-06     	|  4.3948e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  3.2899e-06     	|  2              	|  6.5798e-06     	|  4.3614e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  6.4094e-06     	|  1              	|  6.4094e-06     	|  4.2485e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  3.2e-06        	|  2              	|  6.4e-06        	|  4.2423e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  6.2203e-06     	|  1              	|  6.2203e-06     	|  4.1232e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  3.0403e-06     	|  2              	|  6.0806e-06     	|  4.0306e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  3.0352e-06     	|  2              	|  6.0704e-06     	|  4.0238e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  2.9802e-06     	|  2              	|  5.9605e-06     	|  3.9509e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  5.5293e-06     	|  1              	|  5.5293e-06     	|  3.6651e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  5.3197e-06     	|  1              	|  5.3197e-06     	|  3.5262e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  4.8596e-06     	|  1              	|  4.8596e-06     	|  3.2212e-07     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  4.61e-06       	|  1              	|  4.61e-06       	|  3.0558e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  4.3903e-06     	|  1              	|  4.3903e-06     	|  2.9101e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  3.8398e-06     	|  1              	|  3.8398e-06     	|  2.5453e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  3.6992e-06     	|  1              	|  3.6992e-06     	|  2.452e-07      	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  3.6098e-06     	|  1              	|  3.6098e-06     	|  2.3928e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  3.5902e-06     	|  1              	|  3.5902e-06     	|  2.3798e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  3.58e-06       	|  1              	|  3.58e-06       	|  2.373e-07      	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  3.5604e-06     	|  1              	|  3.5604e-06     	|  2.3601e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  3.4198e-06     	|  1              	|  3.4198e-06     	|  2.2668e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  3.4198e-06     	|  1              	|  3.4198e-06     	|  2.2668e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  3.3807e-06     	|  1              	|  3.3807e-06     	|  2.2409e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  3.1199e-06     	|  1              	|  3.1199e-06     	|  2.0681e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  3.0501e-06     	|  1              	|  3.0501e-06     	|  2.0218e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  2.9197e-06     	|  1              	|  2.9197e-06     	|  1.9353e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  1.3942e-06     	|  2              	|  2.7884e-06     	|  1.8483e-07     	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  2.7204e-06     	|  1              	|  2.7204e-06     	|  1.8032e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  2.6599e-06     	|  1              	|  2.6599e-06     	|  1.7631e-07     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  1.3248e-06     	|  2              	|  2.6496e-06     	|  1.7563e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  2.6301e-06     	|  1              	|  2.6301e-06     	|  1.7433e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  2.5798e-06     	|  1              	|  2.5798e-06     	|  1.71e-07       	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  2.3898e-06     	|  1              	|  2.3898e-06     	|  1.5841e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  2.3702e-06     	|  1              	|  2.3702e-06     	|  1.5711e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  2.2706e-06     	|  1              	|  2.2706e-06     	|  1.5051e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  2.2203e-06     	|  1              	|  2.2203e-06     	|  1.4717e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  2.1597e-06     	|  1              	|  2.1597e-06     	|  1.4316e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  2.0796e-06     	|  1              	|  2.0796e-06     	|  1.3785e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  2.0601e-06     	|  1              	|  2.0601e-06     	|  1.3655e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  1.9707e-06     	|  1              	|  1.9707e-06     	|  1.3063e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  1.8701e-06     	|  1              	|  1.8701e-06     	|  1.2396e-07     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  1.6e-06        	|  1              	|  1.6e-06        	|  1.0606e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  1.5507e-06     	|  1              	|  1.5507e-06     	|  1.0279e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  1.3895e-06     	|  1              	|  1.3895e-06     	|  9.2106e-08     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  1.3597e-06     	|  1              	|  1.3597e-06     	|  9.013e-08      	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  1.3392e-06     	|  1              	|  1.3392e-06     	|  8.8772e-08     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
