jilijeanlouis
Joined 5 years ago
Karma
18
Submissions
50
Member since
Sep 24
Recent submissions
Comment · 4 months ago
Pretty cool !
Comment · 7 months ago
did you try gladia: ranking #1 on STT blind test https://compare-stt.com/
Comment · 7 months ago
same for gladia it's ranked top 1 in the STT blind tests: https://compare-stt.com/
Comment · 7 months ago
Author here. We built this because we kept seeing different word error rates (WER) for the same models depending on who was testing and how.
Normalization rules ended up being a big reason why this was happening, so we decided to release a fully reproducible evaluation framework. You can test it yourself with our full repo.
It includes: Normalization rules we use; Scoring scripts; Dataset coverage (conversational, noisy, multilingual); Full eval pipeline
We also published a detailed comparison using this framework across 8 leading STT providers, 7 datasets, and 74 hours of audio. You can see it here: https://www.gladia.io/competitors/benchmarks
Feedback welcomed!
Comment · 2 years ago
Having worked with Whisper for quite some time now, it's true that hallucinations can be a real pain point. Long pauses between sentences / silence and background noise make it worse, among other factors. Good news is, there are ways to mitigate and solve this.
Comment · 2 years ago
This is really easy to do: it's just an embedding of your voice. So typically like 10/30 sec max of your voice to configure this. You already do a similar setup for faceId. I agree with you, I don't understand why they don't do it.
Comment · 2 years ago
Our API, Gladia, supports speaker diarization. We use a hybrid enterprise-grade ASR system for speech-to-text, with our own version of Whisper at its core, and state-of-the-art open source models for diarization. We process large audio files, and use a proprietary algorithm so that our users are not billed extra for duplicate audio channels, as many other providers do. Hope this helps. There's a free trial if you'd like to test, and here's our blog with more info:
https://www.gladia.io/blog/gladia-speech-to-text-api-speaker...
Comment · 2 years ago
Language detection in the presence of strong accents is, in my opinion, one of the most under-discussed biases in AI.
Traditional ASR systems struggle when English (or any language) is spoken with a heavy accent, often confusing it with another language. Whisper is also affected by this issue, as you noted.
The root of this problem lies in how language detection typically works. It relies on analyzing audio via MFCC (Mel Frequency Cepstrum Coefficient), a method inspired by human auditory perception.
MFCC is a part of the "psychoacoustic" field, focusing on how we perceive sound. It emphasizes lower frequencies and uses techniques like normalized Fourier decomposition to convert audio into a frequency spectrum.
However, this approach has a limitation: it's based purely on acoustics. So, if you speak English with a strong accent, the system may not understand the content but instead judge based on your prosody (rhythm, stress, intonation).
With the team at Gladia, we've developed a hybrid approach that combines psycho-acoustic features with content understanding for dynamic language detection.
In simple terms, our system doesn't just listen to how you speak but also understands what you're saying. This dual approach allows for efficient code-switching and doesn't let strong accents fall through the cracks. The system is based on optimized Whisper, among other models.
In the end, we managed to solve 99% of edge cases involving strong accents, despite the initial Whisper bias there. We've also worked a lot on hallucinations as a separate problem, which resulted in our proprietary model called Whisper-Zero.
If you want to give it a try, there's a free tier available. I'm happy to bounce around ideas on this topic any time; it's super fascinating to me.