dataqa
Joined 5 years ago
Karma
18
Submissions
13
Member since
Sep 22
[email protected]
View on Hacker NewsRecent submissions
Comment · 5 years ago
The open-source project is here: https://github.com/dataqa/dataqa. I have just released a feature which I have been working on for a while to solve a problem which I've seen a lot in industry: how to map entities found in text to large knowledge base ontologies.
Comment · 5 years ago
There is a tutorial on the site where you use rules to extract mentions of side effects from forum posts: https://dataqa.ai/docs/tutorials/medical_side_effects/ner_me.... You can use this tool for NER.
Comment · 5 years ago
Thanks for sharing! It looks very interesting. From a brief check, they do not seem to be UI-based like dataqa (although you can use it in a notebook), they do not offer a search engine and they are probably one level of abstraction below dataqa. You can do some of the stuff dataqa does but would need to code. Some of the rules offered by dataqa rely on complex operations with regular expressions, and are not so easy to program yourself.
Comment · 5 years ago
Thanks for the tip! The tool can deal with larger documents, so you're right that using a folder would be better in that case.
Comment · 5 years ago
I have seen first hand at small and large companies how problems have been tackled with ML without trying a simple rule or heuristic first. And then, further down the line, the system has been compared to a few business rules put together, to find that the difference in performance did not explain the deployment of an ML system in the first place.
It's true that if your rules grow in complexity, this might make it harder to maintain, but the good thing about rules is that they tend to be fully explainable, and they can be encoded by domain experts. So the maintenance of such a system does not need to be done exclusively by an ML engineer anymore.
Here is where I insert my plug: I have developed a tool to create rules to solve NLP problems: https://github.com/dataqa/dataqa
Comment · 5 years ago
Hey HN!
After working in ML for more than a decade, I became frustrated over time with the lack of tools to create baselines using simple rules and heuristics. It is well known that most business problems can achieve decent baselines using only heuristics. So this is why I have just open-sourced DataQA, a rules-based labelling tool for NLP:
- Quick labelling: You can create complex rules using regular expressions to help you label your text faster.
- Search engine: DataQA also ships with a search engine (local elasticsearch database) so you can search your documents.
- Easy installation: Only need to install a single python package!
- Easy use: upload your data as csv files.
- Privacy: No data ever leaves your computer.