Simplifying DINO via Coding Rate Regularization
Автор: LuxaK
Загружено: 2025-02-20
Просмотров: 28
This paper introduces SimDINO and SimDINOv2, simplified versions of the DINO and DINOv2 self-supervised learning models. The authors streamline the complex training pipelines of the original models by replacing numerous empirical design choices with an explicit coding rate regularization term in the loss function. This simplification enhances robustness to hyperparameter variations and network architectures. Remarkably, the resulting SimDINO models not only simplify the training process but also achieve superior performance on downstream tasks. Experiments demonstrate that SimDINO and SimDINOv2 exhibit improved stability, ease of optimization, and higher-quality representations compared to their DINO counterparts. The research highlights the value of simplifying deep learning pipelines for improved empirical practice in vision self-supervised learning.
paper - https://arxiv.org/pdf/2502.10385v1
subscribe - https://t.me/arxivdotorg
created with NotebookLM
Доступные форматы для скачивания:
Скачать видео mp4
-
Информация по загрузке:
