Ask HN: Non-tech guy looking for unstructured data startup advice
I have an investment background but am interested in creating a new finance product for internal use. My idea is to take all historical filings that public companies submit to the SEC (as well as a live feed of new filings) and basically create a Bloomberg for mostly qualitative information that the big financial software companies do not analyze.
For example, I would like to take a given company's entire document history (almost 20 years of text files) and create an application that can determine whether the company has ever received a material weakness in its internal controls notice, the dates received/outstanding, the reasons provided, and when/if it was finally resolved or not. There are about 50-80 other conditions like this I would like to discover through an application, but what's tricky is that companies can use different language to describe these things.
I have absolutely no idea where to start with a project like this, but am excited to learn some new tricks. Am wondering about how to best extract unstructured data I want, best ways to populate and organize a DB (presumably NoSQL), etc. I realize this might sound comically general and naive, but figured there might be some folks out there with good experience working with unstructured data and document-oriented databases. Any thoughts are GREATLY appreciated! Thanks.
For example, I would like to take a given company's entire document history (almost 20 years of text files) and create an application that can determine whether the company has ever received a material weakness in its internal controls notice, the dates received/outstanding, the reasons provided, and when/if it was finally resolved or not. There are about 50-80 other conditions like this I would like to discover through an application, but what's tricky is that companies can use different language to describe these things.
I have absolutely no idea where to start with a project like this, but am excited to learn some new tricks. Am wondering about how to best extract unstructured data I want, best ways to populate and organize a DB (presumably NoSQL), etc. I realize this might sound comically general and naive, but figured there might be some folks out there with good experience working with unstructured data and document-oriented databases. Any thoughts are GREATLY appreciated! Thanks.