Validation Reloaded - Elevate Your Model Validation with Advanced Toolkits
Keywords: Metrics, Rankings, Validation, AI, Toolkit, Open-Source
TL;DR: In our blog post on Medium, we will introduce you to two toolkits: Metrics Reloaded for selecting appropriate metrics for your research problem and Rankings Reloaded for robustly ranking your method against other models or hyperparameter settings.
Abstract: Validating artificial intelligence (AI) algorithms is often seen as something boring - something that you do on the side while you focus on developing new models to improve your AI even more. But if you want to improve your model precisely, you need to measure that improvement. The only way to do that is through proper, problem-tailored validation. In current research, there is an overemphasis on developing new AI architectures, but a lack of rigorous comparisons and thorough validation. Validation may be underestimated, but is essential to prove that your algorithm does what it is supposed to do.
In recent years, theory has been developed to address these issues in validation. Large groups of researchers have worked on problem-aware metric recommendations and advanced analysis and visualization tools for benchmarking (note: there are other outstanding publications along these lines, but we will focus on these two topics). While these were great achievements towards better research practices, the theory is not easy to understand. That’s why we have been working on making the theory easy to use for the community by creating online toolkits that are easy to use. The good thing is that there is no need to worry or fully understand the theory - you are automatically guided through the process :) In this blog post, we will introduce you to two toolkits: Metrics Reloaded for selecting appropriate metrics for your research problem and Rankings Reloaded for robustly ranking your method against other models or hyperparameter settings.
___________________________________
Original Medium blog post can be found at https://medium.com/miccai-educational-initiative/validation-reloaded-elevate-your-model-validation-with-advanced-toolkits-2d67d07d2460
Please note:
The original Medium blog post was converted into a PDF for submission.
The original blog post contains both GIFs and videos, which cannot be displayed correctly in a PDF file. For the PDF, we have replaced the GIFs with screenshots and added links to the videos.
For better readability, we recommend reading the original blog post under the link above.
Website: https://medium.com/miccai-educational-initiative/validation-reloaded-elevate-your-model-validation-with-advanced-toolkits-2d67d07d2460
Submission Number: 2
Loading