TonyHaenn
Joined 5 years ago
Karma
6
Submissions
7
Member since
Apr 2
Recent submissions
Comment · 2 years ago
I think the software is the easy part (and that's still hard). The harder part is finding a problem that people will pay you to solve. I'd suggest for each of your ideas, you try to validate that there's a group of people that you can reliably find and talk to (marketing) about the problem they have. Only build something once you have confidence you can sell it.
I like the book the Mom Test. Easy read, and gives some really practical tips on how to talk to people about their problems. Focus on the problem first and wait to build something until later.
Comment · 2 years ago
Depends on what you're aiming for. For my use case, I'm aiming for the feeling of talking to another human. I built an iOS app for little kids to call Santa. Low latency was important. Now I'm working on a mock interview experience; same deal, needs to feel like the real thing.
Re: contextualizing the filler. No, but it's a good idea :) This thread made me think there's a way to generate one on the fly based on the first part of what the person has said. The challenge though is it seems to me that filler phrases usually relate to what the person said last, not first.
Comment · 2 years ago
Nice writeup! Super interesting that we both took different paths, but ended up with similar latencies.
I built a real-time conversation platform in Elixir. I used the Membrane framework to coordinate amongst the STT, LLM and TTS steps. I also ended up with latency in the ~1300 ms range.
I found research that says the typical human response time is 250 to 300 ms [0] in a conversation, so I think that should be the goal.
For my solution, some of the things we did to get latency as low as possible: 1. We stream the audio to the TTS endpoint. If you're transcribing as the audio comes in, then all you care about is the tail latency (the time between when the audio ends and the final transcript arrives). That helped a bunch for us. Google is around 200 ms with this approach.
2. Gpt 3.5 still has a time to first token of ~350 to ~400 ms. I couldn't find a way around that. But you can stream those tokens to ElevenLabs and start getting audio faster which helps.
3. ElevenLabs eats us most of the latency budget. Even with their turbo model their latency is 600-800 ms according to my timings. Again, streaming the words in (not tokens) and calling flush seemed to help.
The key I found was to cover up the latency. We respond immediately with some filler audio. The trick was getting the LLM to be aware of the filler audio text and continue naturally from that point
[0] https://journalofcognition.org/articles/10.5334/joc.268#
Comment · 2 years ago
Thank you!
React Native still has some sharp corners. The expo tooling is pretty rad, but I could never get the Android version to build.
Hopefully LiveView Native has a path around some of those issues. I agree it’d be pretty cool to have one language end to end.
Comment · 3 years ago
I’ve personally always learned better when it’s in service of building something. Pick a language (e.g. python) and build something you’d find useful or interesting.
Comment · 5 years ago
Lola | https://www.lola.com | Remote or Boston, MA | FE Engineers, DevOps
Last year the pandemic wiped out a bunch of the travel sector. Lots of travel companies went under or were acquired. Lola moved fast and expanded into the FinTech sector. We launched an MVP of our new Spend Management platform in three months, and six months after launch we're up 150% MoM in customer spend.
Our early customers rave about what we've built, and we're really excited for whats up next. We're entering a crowded space with some competitors that have valuations over $1B -- but we're winning deals against them when we go head to head. Our approach to the customer problem is unique, and CFOs of SMBs get excited when they realize what we're building. It's a fun place to work -- we don't do death marches, and the engineering team has a strong voice in how we build product.
We've got a strong leadership team in place. Our CTO is Paul English, the former co-founder of Kayak, and Mike Volpe, the CMO of Hubspot, CyberReason and a few others. They've helped other startups get big, and we're all really confident in what the rest of the year is going to bring.
FE https://jobs.lever.co/lola/0ced61e7-eac8-4c66-a1d8-4eee88d04...
DevOps https://jobs.lever.co/lola/fe217951-43a8-455d-8456-e7a31acf5...
Product nerd. I like to build and I'm super curious about technology.
Things I've built: https://callsanta.ai https://interviewgenius.io https://smartvoxlabs.com