ryanwaldorf

Oh that's really interesting! I imagine there could be a reason for it, for instance the data is distributed differently in the micropartitions so different where values could result in different data lookup patterns as you may skip more/less blocks. But overall this makes a lot of sense!

Really interesting! Could you clarify what difference having two different query_parameterized_hash for similar queries is? Is there a performance hit?

Totally agree. In my last job I was able to create my own ETL jobs as a PM to get data for my own analyses and figured out a fairly minor configuration change could save us $10M per year. It was from one of many random ETL jobs I created myself out of curiosity that, if I had been forced to rely on other people, I may not ever have created.

What motivated this: I kept hitting token limits when working with Claude on tasks with many parts. For instance, Claude would recommend three different areas of code to work on, but by the time I started on the second area, I'd get token limit warnings. This happened because Claude needs to process the entire conversation history with each response, even though the discussion of the first area is often unrelated to the second.

This extension lets you fork the conversation at any point, preserving all context and files up to that point while starting fresh with the token count. You can simply copy and paste it into a new chat or (download it as a file to attach) and continue your chat. Your conversation continues from the forked point as if it were the same conversation.

I've really enjoyed using Claude projects to help generate code drafts. However, one thing that's been painful is uploading new files versions after changes. I'd have to separately go through each folder in my project to upload the files again so Claude could have the most recent versions. It only took a few minutes each time....but it was a bit annoying.

As a result I built a file organizer for Claude projects. It selectively copies files based on extensions and .gitignore rules, organizing them into a target folder for easy uploading. It commits all the files in that folder to a git repository so they're not lost, deletes them, and then copies the updated files to that folder. This allows you to upload your latest file states to a Claude project without having to manually click through all your folders to select and upload specific files.

Hope y'all find this useful!

just replied to chrisjc that I created a branch with externalbrowser auth and pushed it to github. would you mind taking a look and letting me know if it works for you? would love to get your feedback as I don't have SSO set up in my account to test this myself

This makes sense, thank you!

Ah gotcha! Do you have a use case where you'd look to remodel/transform the data between warehouses?

Thank you! Will take a look at this over the next few days

Would that be to support SSO? If so, I haven't planned it yet but I could. Would that make this more useful to you?

That's really interesting! Could you tell me a bit more of what you're thinking? I'm not the most familiar with SQL Mesh and the typical workflows there.

I wanted to start with duckdb since it's really an incredibly powerful tool that people should try out. The performance you can get on analytical queries running on your local compute is just really impressive. And with snowflake streams you can actually stream live data into it without changing anything about your existing data. On why not other databases, I wanted to focus on OLAP to start as there are already other great tools like DLT that help you load data from OLTP sources like postgres and mysql to OLAP sources already, but OLAP to OLAP is pretty rare.

Have you run into a use case for streaming data between data warehouses yourself yet? If so which warehouses?