laalshaitaan

ai slop again

that on the customer's side to redact the PII as you are the ones sending it and you'd know better than us

i get the push, most teams come to us after they've done/tired of the claude running analysis thing manually and want a pro-active thing.

we're also targeting conversation first use cases and for them this serves as their everything custoemrs. we obv do not sell the fancy part, idts that sells anymore lol. lot of our queries come from our slack app/mcp.

its fascinating how the toyota example comes up anywhere, its so good!

wdym by modelling the messages and conversations though? i lose you a bit there! for the crm approach, i do think it'll be a problem at some point right now.

the replacing budge is an interesting piece, we did not think of it yet, yes let me dm you on x!

a lot of our customers want a daily morning report on slack & flag things instantly rather than to wait for a week so thats why we keep it realtime

it gets hard when you need this continuously across lots of chats/calls, with metadata, changing clusters, going deeper into a user journey, etc. the LLM call is just one part of it lol

we're keeping it useful every week, finding out insights that the teams can extract value out of, work with them to understand users better.

the procurement what we've seen is v similar to how one would have for any analytics product? and we're selling this to companies when/once it becomes someone's job to do this

i agree w you for smaller teams tbh. if you have a few hundred convos and someone can maintain scripts/prompts, hand-rolling is probably fine.

it becomes less tiny when it’s 10k+ msgs/week, long voice calls, metadata, changing clusters, retention/redaction, and the team wants this continuously without maintaining another internal tool.

but i'd still wanna know how you'd do this with existing tools before llms?

im hearing this for the first time and damn! i just told this to my cofounder/cto and he said hes gonna give this a shot in the coming days.

damn, i read bayesian in statistics like years ago, never thought itll come back this way

seems ai slop

oh gotcha, yeah happy to chat more! yeah DMing

i love this comment bec we started as analytics for mcp servers haha! we then expanded to conversations bec thats where most mcp servers were being used lol.

we havnt figured out yet how to do the b2b2b kinda thing where we surface insights for a multi-tenant sort of approach but i've gotten this now twice in the last hour so happy to chat.

what would you want them to see first though, are these semantic insights like we do today or more around deterministic tool calls/etc metrics?

i like these kinds of critiques, we don’t think conversation logs or analysis on top of it is alone enough to replace observability or evals. imo they answer diff questions for diff use-cases.

we're betting that there is a TONNN of product signal buried in conversations that observability misses, esp around like raging, writing in all caps, repeated prompts, frustration loops, and subtle hidden feature demand. thats also why we use per-customer taxonomies instead of a shared one. evals will still be needed.

the root cause is harder, especially in more mature agents. we're using this more as a discovery layer for evals or even just whats happening kind of things, then letting teams go deep into the actual conversations and decide what to take action upon

yea, at our volume which we still consider small as we've been able to figure out a way with llms & embeddings, its still fine. + we onboarded a voice ai company with more than 2 hour calls and thats when it was super hard to solve since there were so many elements to consider.

model drifting is something a lot of folks do face after 5th/6th turn as per my understanding and it usually the median, how did you tackle it if you have yet?

also yea, thats why we went for a per customer taxonomy than a general one, yeilded better results + easier to improve upon.

okay nice, also is it safe to assume you do once a fortnight releases then? like look at the last week's data then use it for product decisions the coming week?

also have you updated/made any changes to this skill that has improved it significantly?

and anything you hate/wish it had as of today? wanna learn if there's any painpoints around this? is it keeping the skill updated, getting useful signal from the clusters, or turning the findings into something the rest of the team can actually act on?

oh damn, can you share what the skill is actually doing for you like on a daily basis: is it creating clusters, scoring known issues, or finding new patterns? and what data points are you giving it to if any w the messages?

usually a boundary for us is usually where a skill/claude analysis needs to maintain/make changes/pass it to an agent as a workflow

for the pricing we're still learning and building as many custom features/requirements as possible bec we wanna make sure we deliver way more value than what we charge today.