dnth
Joined 3 years ago
Karma
59
Submissions
23
Member since
Feb 15
Hello, I am Dickson. I share practical deep learning tips and tricks at https://dicksonneoh.com/
View on Hacker NewsRecent submissions
Comment · 3 years ago
We released a free tool (fastdup) where you can use to cluster image using DINOv2 embeddings.
Comment · 3 years ago
fastdup is a tool that let you gain insights from a large image/video collection.
It lets you identify image duplicates, video duplicates, wrong labels, outliers, corrupted data, and image clusters.
fastdup is -
Unsupervised: fits any visual dataset. Scalable: handles 400M images on a single machine. Efficient: works on CPU (even on Google Colab with only 2 CPU cores!). Low Cost: can process 12M images on a $1 cloud machine budget.
Comment · 3 years ago
The authors at fastdup ran an analysis on LAION 400M and Imagenet21K. Here's what they found.
LAION 400M
> 60M duplicates. > 962K broken images. > Various label discrepancies.
ImageNet21K
> 1.2M duplicate images. > 104K train/val leak.
fastdup GitHub repo - https://github.com/visual-layer/fastdup
Comment · 3 years ago
Even pros have dark, blurry & duplicate shots. But disorganization can make it hard to find those special memories. Let's fix that.
Comment · 3 years ago
Update - After some code optimization, I got the inference time down to below 100ms. The lowest I got on my Pixel 3 XL is 37ms!
https://dicksonneoh.com/portfolio/pytorch_at_the_edge_timm_t...
Comment · 3 years ago
You're welcome! I'm not sure if I'm the right person to advise on this. But this idea is also known as federated learning right?
Comment · 3 years ago
With flutter you can also build web apps.
Comment · 3 years ago
Note yet! But I heard things are going to be a lot faster in 2.0. Have you tried?