Machine learning is pervasive in the new products targeted at cybersecurity and infosec. From malware to anomaly detection among billions of logs, it's been part of the demand of my customers for at least four years now. Most of the largest companies are already running and testing a series of novel proprietary or custom in-house systems.
The main differences with straight 'supervised' machine learning are the lack of labels, and the unreal volume of data (cheaply produced by machines, in machine time, we're talking microseconds to milliseconds here). So unsupervised learning is king in this domain, and often security operators have to keep an eye on and interpret the results. Another difference with other fields is that datasets are rare, mostly because of privacy as the logs can be very revealing. For this reason the market exhibits a lock that cannot be overcome by everyone I believe. Basically, you sort of need to be in the place already.
Here is a recent very high level survey on the topic (a TC report, not that bad for once), https://techcrunch.com/2016/07/01/exploiting-machine-learnin...
Here are two of our own Open Source tooling we use the most in application:
- very efficient C++ map-reduce feature generation for logs (for ML and analytics): https://github.com/soprasteria/cybersecurity-miw
- machine learning / deep learning server: https://github.com/beniz/deepdetect
The ML cybersecurity + infosec field is still young, but moving very fast, a lot of new startups and (somewhat opaque) products.
The main differences with straight 'supervised' machine learning are the lack of labels, and the unreal volume of data (cheaply produced by machines, in machine time, we're talking microseconds to milliseconds here). So unsupervised learning is king in this domain, and often security operators have to keep an eye on and interpret the results. Another difference with other fields is that datasets are rare, mostly because of privacy as the logs can be very revealing. For this reason the market exhibits a lock that cannot be overcome by everyone I believe. Basically, you sort of need to be in the place already.
Here is a recent very high level survey on the topic (a TC report, not that bad for once), https://techcrunch.com/2016/07/01/exploiting-machine-learnin...
Here are two of our own Open Source tooling we use the most in application:
- very efficient C++ map-reduce feature generation for logs (for ML and analytics): https://github.com/soprasteria/cybersecurity-miw
- machine learning / deep learning server: https://github.com/beniz/deepdetect
The ML cybersecurity + infosec field is still young, but moving very fast, a lot of new startups and (somewhat opaque) products.