mwsherman
Joined 17 years ago
Karma
3,607
Submissions
667
Member since
Jul 21
https://clipperhouse.com/
View on Hacker NewsRecent submissions
Comment · 3 months ago
I recommend starting with SWAR [1] before SIMD. Our registers are typically 64 bits, and one can try out SIMD patterns without taking a dependency on particular hardware.
This will only be effective if the data you’re working on is smaller than 64 bits. If you’re working with bytes, for example, you might get 8x parallelism.
Comment · 9 months ago
While this article is about perf — and trading off semantic precision by design — there is a Unicode standard for sentence boundaries, may be interesting: https://www.unicode.org/reports/tr29/#Sentence_Boundaries
I implemented the sentence boundaries, but also thought that the notion of a “phrase” might be useful for such applications: https://github.com/clipperhouse/uax29/tree/master/phrases
Comment · 10 months ago
Shameless plug, if one wishes to track down allocations in Go, an allocations explorer for VS Code: https://marketplace.visualstudio.com/items?itemName=Clipperh...
Comment · 10 months ago
I’ve found C#’s frozen dictionary to be useful: https://learn.microsoft.com/en-us/dotnet/api/system.collecti...
It’s optimized for fast reads in exchange for expensive creation.
Comment · 1 year ago
That’s true, converting in either direction will typically allocate. Which it must, semantically.
One can use unsafe for a zero-copy conversion, but now you are breaking the semantics: a string becomes mutable, because its underlying bytes are mutable.
Or! One can often handle strings and bytes interchangeably with generics: https://github.com/clipperhouse/stringish
Comment · 1 year ago
There is mention of how len() is bytes, not “characters”. A further subtlety: a rune (codepoint) is still not necessarily a “character” in terms of what is displayed for users — that would be a “grapheme”.
A grapheme can be multiple codepoints, with modifiers, joiners, etc.
This is true in all languages, it’s a Unicode thing, not a Go thing. Shameless plug, here is a grapheme tokenizer for Go: https://github.com/clipperhouse/uax29/tree/master/graphemes
Comment · 1 year ago
In Go, string effectively serves as a read-only slice, if we are talking about bytes.
ReadOnlySpan<T> in C# is great! In my opinion, Go essentially designed in “span” from the start.
Comment · 1 year ago
A Go allocations explorer for VS Code. Extension: https://marketplace.visualstudio.com/items?itemName=Clipperh...
Comment · 1 year ago
Shameless plug, you may wish to do Lucene-style tokenizing using the Unicode standard: https://github.com/clipperhouse/uax29/tree/master/words
Comment · 1 year ago
I move between Go and C#. I wrote a zero-allocation package in Go [1] and then ported to C# — and the allocations exploded!
I had forgotten, or perhaps never realized, that substrings in C# allocate. The solution was Spans.
Notably, it caused me to realize that Go had “spans” designed in from the start.