lkowalcz
Joined 13 years ago
Karma
49
Submissions
29
Member since
Apr 22
Recent submissions
Comment · 6 years ago
At IEX, we use kdb.
Here’s a blog post from a former employee at a new startup comparing kdb against some alternative databases for common operations on financial data: https://medium.com/prooftrading/selecting-a-database-for-an-...
Comment · 8 years ago
I think IEX has a free API with historical and real-time stock info: https://iextrading.com/developer/docs/
Comment · 9 years ago
I think it's appropriate to restrict usage of the word "encryption" to methods that come with some justification for their security.
I don't think that doing this stifles innovation (In fact, I think requiring crypto innovations to have security justifications is probably better overall for innovation in the field)
Comment · 9 years ago
I've never heard of neural encryption before, do you have a good source for me?
From the Wikipedia page for "Neural cryptography", it seems like there's some success in using NN's for cryptanalysis, but not for constructions...
Edit: Do you mean the Google GAN experiment? (https://arxiv.org/pdf/1610.06918v1.pdf) Ahh ok, well at least for this there looks like an attempt at defining a security model (security against some other NN). I don't really believe the security model is realistic (how do we know NN's are really that effective as adversaries?), but at least there is a model, so calling that "encryption" sits somewhat better with me. I'm pretty sure this is not what's being used by Numerai since it seems like it would not result in ciphertexts with the structure necessary to perform ML operations on.
Edit2: Maybe you're right and it is more advanced. In any case, as a crypto nerd I wish they would disclose what they are actually doing / the rationale instead of tantalizingly suggesting that they have made (what would be) a breakthrough in a practical use case of advanced encryption schemes, but not saying how.
Comment · 9 years ago
Thanks for that link! I was never able to find any details from someone who works for Numerai (or claims to at least)
I still don't think it's fair to market their method as comparable to Aslett's scheme or "standard" notions of homormorphic/order preserving encryption, no matter how "chill" they are :)
Comment · 9 years ago
I suspect they might just be anonymizing their features (removing any labels), then normalizing them to [0,1].
Comment · 9 years ago
Worth pointing out that Numerai actually doesn't use encryption in any standard sense (including structure-preserving encryption), but instead seems to be using some heuristic method of obfuscating their data.
Their (closed-source) method of obfuscating their data apparently does have the property that it preserves the structure of the data, but calling it "structure-preserving encryption" is misleading imo since it risks confusing it with standard notions of encryption and structure-preserving encryption which have much stronger security guarantees.
(Their marketing seems to encourage this conflation by, for example, citing academic advances in standard notions of homomorphic encryption and SPE and implying that these advances have enabled Numerai's technology)
https://medium.com/numerai/encrypted-data-for-efficient-mark...
Comment · 9 years ago
Most NSF grants and the like are for PhD students (the university still welcomes the funds they bring in, but they are a relatively small portion of income).
Universities make much more money off their Masters students, most of whom are international, and pay full tuition (or someone does for them).
Comment · 9 years ago
There is a huge difference in how universities treat their Masters vs. PhD students. Indeed, universities are not making loads of money off international PhD students (most receive free tuition, a stipend, etc). However, international Masters students are cash cows for most universities (most pay full price -- or someone does for them).
Comment · 10 years ago
Ah ok, sure. I wouldn't call something like a linear transform on anonymized features "encryption" (more like obfuscation?), but I guess it's good marketing in that it lets them associate with the "recent advances in [real] homomorphic encryption"
Comment · 10 years ago
I'm almost positive it's not using homomorphic encryption (at least in any real sense). It's misleading how they seem to suggest that they are, though.
Comment · 10 years ago
The encryption here is being done by "adding 20" / "multiplying by -0.5"?
Given this "encrypted" X , y dataset, I could easily find the unencrypted version... (even if I don't know 20 or -0.5, this still reveals so much of the structure that I don't believe it provides any real protection against anything except the most lazy attackers)
https://www.cs.columbia.edu/~luke/