IvanVergiliev
Joined 14 years ago
Karma
86
Submissions
27
Member since
Mar 16
Recent submissions
Comment · 2 years ago
Comment · 2 years ago
Just an FYI in case you haven’t read this: Recursive Make Considered Harmful [1]
Comment · 3 years ago
You can use a “Custom Study Session” [1] for this. I tried it recently and it was fairly decent.
One possible way to get there on mobile: 1. Start reviewing a deck. 2. Click the gear button. 3. Choose “Custom study” and select one of the options. 4. If studying tagged cards, go tag some cards via the Browse view.
[1] https://docs.ankiweb.net/filtered-decks.html#custom-study
Comment · 5 years ago
Definitely. One of the primary benefits we get out of Spark is the ability to decouple storage and compute, and to very easily scale out the compute.
Our main Spark workload is pretty spiky. We have low load during most of the day, and very high load at certain times - either system-wide, or because a large customer triggered an expensive operation. Using Spark as our distributed query engine allows us to quickly spin up new worker nodes and process the high load in a timely manner. We can then downsize the cluster again to keep our compute spend in check.
And just to provide some context on our data size, here's an article about how we use Citus at Heap - https://www.citusdata.com/customers/heap . We store close to a petabyte of data in our distributed Citus cluster. However, we've found Spark to be significantly better at queries with large result sets - our Connect product syncs a lot of data from our internal storage to customers' warehouses.
Comment · 5 years ago
Post author here. Let me know if you have any questions!
Comment · 5 years ago
Just responded to the parent comment as well - there's an additional mutable argument to the real `transform` method so it's unsafe to invoke it directly without first checking if the tree is convertible.
Comment · 5 years ago
Good question, the simplified example doesn't make this clear.
The real implementation has a mutable `builder` argument used to gradually build the converted filter. If we perform the `transform().isDefined` call directly on the "main" builder, but the subtree turns out to not be convertible, we can mess up the state of the builder.
The second example from the post would look roughly like this:
val transformedLeft = if (transform(tree.left, new Builder()).isDefined) {
transform(tree.left, mainBuilder)
} else None
There's a more detailed explanation in the old comment to the method: https://github.com/apache/spark/pull/24068/files#diff-5de773... .
Comment · 6 years ago
Exclude constraint on a GiST index?
Comment · 6 years ago
Cool, I’ll take a read.
Just to clarify something in case it’s not obvious from my other comments. I’m not arguing for SRS as a replacement to all other forms of learning. You still need the extensive reading, problem solving, experimenting with a programming language.
However, I’ve found SRS to be a great addition to the methods above. For example, I haven’t found the methods above to give you long-term retention on their own. (And I’ve done a lot of problem solving.) Math is also a lot about building up the level of abstraction, and SRS can help with spacing the practice of lower-level concepts so you can more easily apply them to more complicated ones.
I’ve never been a fan of memorization in the past. However, I’ve found that:
1. Memorization (as in knowing foundational facts and being able to recall them efficiently) is actually pretty useful, as much as I didn’t want this to be true.
2. SRS can be used for thinking + deriving the answer to a card in addition to just memorization. It just gives you the right timing to do so.
Comment · 6 years ago
I’ve done some cloze deletions for math and related things, but I generally feel like having almost the whole proof in the prompt gives me too many cues. It often leaves me thinking that I indeed wouldn’t be able to come with the answer with fewer hints.
What I’ve tried the last couple of attempts is to “chunk” the proof (also terminology touched on in Barbara Oakley’s course) so that I end up with a question that’s something like “what’s the high-level idea / approach in the proof for X?”. That card would likely require an understanding of some underlying concepts or “chunks”, so I add questions for these too until I get to something that’s less abstract and easier to rederive.
I’m still not 100% confident if this will work well when these particular cards get into the 6-month range or so, and they start showing up at completely unrelated times. My main concern is that if I’ve forgotten some idea from “the middle”, it would be hard to reason about cards that build up on top of that.
Comment · 6 years ago
> it's a lot better to use those connections instead of drilling it in a decontextualized fragments via SRS.
Or, you can use SRS for "spaced repetition" of making these connections. That is, instead of treating it as rote memorization, use it for the timing effects. When I see a card about X1 which is part of a larger concept Y, I don't think "what was the exact answer to X1, which I remembered without any understanding and will just recite now?".
Instead, I often think "how do I come up with the answer to X1 right now? how does it connect to the larger concept Y?". Even better, if I've recently seen card X2 about the same concept, I might think "how does X1 relate to X2, which I just saw recently?". Sometimes, this actively helps you to make new connections. Of course, you need to explicitly make an effort to do so, yourself. If you practice pure recall only, that's what you'll get from SRS.
As another commenter mentioned, it's a false dichotomy.
Comment · 6 years ago
I'm a big fan of the idea of using Spaced Repetition for this. The idea is that it allows you to both:
- be able to keep your knowledge / understanding of an area around for the long term; but also
- be able to gradually build up your understanding by first committing the fundamentals to memory, and then using that to build up your level of abstraction and get to the more complex ideas and principles.
Michael Nielsen has written fairly extensive explanations of two slightly different approaches in:
- Using spaced repetition systems to see through a piece of mathematics [1]
- Augmenting Long-term Memory [2]
I've only used this particular approach for a handful of subjects so far - indeed, it seems to just take time to build a high-quality, long-term understanding of a thing. However, I've been pretty happy with the process so far.
[1] http://cognitivemedium.com/srs-mathematics [2] http://augmentingcognition.com/ltm.html
Comment · 6 years ago
I’ve found it extremely useful to keep a detailed log of my thoughts and ideas as I’m working on a problem that requires focus. It’s like a thread dump or memory dump of my thinking. Then, if I get interrupted for whatever reason, I can easily go back to the notes and “restore” from the thread dump.
This is a pretty good blog post I found on the subject: http://faq.sealedabstract.com/uninterruptible_programming_su... .
I’ve found various side benefits in addition to being able to focus in shorter time windows. For example:
- it’s useful for dealing with interruptions that are part of work too - e.g. if you’re helping teammates with different projects, or have to switch contexts for other reasons.
- it can be useful as an artifact of work. For example, you’ve spent a lot of time debugging a weird issue and you’re still not making progress, so you can use a second set of eyes. You can share your work notes with a coworker so they can immediately know what you’ve tried, what worked or didn’t, etc. In that context, I like to think of it as “offline pair programming”.
Still super useful when you have no other options though.