seanlaff

The ramdisk that overflows to a real disk is a cool concept that I didn't previously consider. Is this just clever use of bcache? If you have any docs about how this was set up I'd love to read them.

Does duckdb support remote-duckdb as a storage engine? Seems like a way to support distributed duckdbs. Ducks all the way down? :)

Oh! I understand, thanks for walking through that. Yes very terse compared to the equivalent react :)

Would we expect the list to re-render once the fetch finishes? This may just be years of react poisoning my brain

    const Stargazers = () => {
      const stargazers = van.state([]);
      fetch(`https://api.github.com/repos/vanjs-org/van/stargazers?per_page=5`)
        .then((r) => r.json())
        .then((r) => stargazers.val = r);
      
      return ul(
        stargazers.val.map((s) => li(s.login))
      );
    };

Hmm, maybe! Is that the idiomatic way to do async? I was thinking something along the lines of this, which react devs will have written a variation of plenty of times :)

    import { useEffect, useState } from "react";

    const GithubUser = ({ login, avatar_url, html_url }) => (
      <div>
        <img src={avatar_url} style={{width:40, height:40}}/>
        <a href={html_url}>{login}</a>
      </div>
    );

    export function App() {
      const [stargazerResp, setStargazerResp] = useState([]);
      const [repo, setRepo] = useState("vanjs-org/van");

      useEffect(() => {
        fetch(`https://api.github.com/repos/${repo}/stargazers?per_page=5`)
          .then((r) => r.json())
          .then((r) => Array.isArray(r) ? setStargazerResp(r) : setStargazerResp([]));
      }, [repo]);

      return (
        <div className="App">
          <input value={repo} onChange={(e) => setRepo(e.target.value)} />
          <ul>
            {stargazerResp.map((s) => (
              <li key={s.login}>
                <GithubUser
                  login={s.login}
                  avatar_url={s.avatar_url}
                  html_url={s.html_url}
                />
              </li>
            ))}
            </ul>
        </div>
      );
    }

This is cool, though I think a table-stakes example that is missing is how to do a network request. I see the stargazers example but that entire component is awaited, which doesn't mirror the common case of async fetch in response to user input, in which the response is fed to sub-components.

The stargazer example leaves me with questions like- are components both async and non-async? What if the component re-renders due to other state changes, is my network request fired more than once? Do I have a "component coloring" problem where once one subcomponent is async, the entire parent hierarchy has to be?

Im sure there's answers to these questions if I read the docs, but as a curious passer-byer an example mirroring this common ui pattern would answer a lot of questions for me!

Ah yup, looks like @saurik pin-pointed my original source (iirc I slightly tweaked the jq, but you get the idea). Please do spread the idea- I would love capability native like this in gitlab.

Im surprised there's still no "trace" view of pipeline execution, especially with how prevalent DAG pipelines have become. Somewhere on the internet I found this handy jq oneliner that will convert a pipeline into a format you can drag and drop into chrome://tracing/ to figure out where your bottlenecks are

  curl "https://${GITLAB_URL}/api/v4/projects/${GITLAB_PROJECT}/pipelines/${GITLAB_PIPELINE}/jobs?per_page=100&private_token=${GITLAB_TOKEN}" | jq 'map([select(.started_at and .finished_at) | {name: (.stage + ": " + .name), cat: "PERF", ph: "B", pid: .pipeline.id, tid: .id, ts: (.started_at | sub("\\.[0-9]+Z$"; "Z") | fromdate \* 10e5)}, {name: (.stage + ": " + .name), cat: "PERF", ph: "E", pid: .pipeline.id, tid: .id, ts: (.finished_at | sub("\\.[0-9]+Z$"; "Z") | fromdate \* 10e5)}]) | flatten(1) | .[]' | jq -s > "${GITLAB_PIPELINE}-trace.txt"

Is there more behind the scenes than just the `timestamp | json` table? From what I understand, any query in clickhouse against that involving a filter would require a full table scan

Cities look so much more fun to explore when pedestrian-focused

Many outlets gloss over the n+1 query problem, but I find it to be a major shortcoming of the gql spec. Sure there are solutions, but they are not very ergonomic. In one of our products we found simple requests blowing out to 100s of downstream calls to the legacy REST APIs. The pain in optimizing the GQL resolving layer negated almost any benefit of the framework as a whole.

Im happy GQL brought api-contract-type-saftey to a wider audience, but it is similarly solved with swagger/OpenApi/protobuf/et al.

Folks may be interested in dgraph, which exposes a gql api and is not victim to the n+1 problem since it is actually a graph behind the scenes.

I’ve been curling this for years but was not aware of the new v2 api that shows hourly ascii charts. Looks sweet!

Neat! How do you handle cross-geo latency? I know google has to rely on precise clocks to make this work in spanner

This depends on row-based binlog replication, correct? Has netflix had to deal with systems with statement-based replication?