simonblanke
Joined 5 years ago
Karma
37
Submissions
28
Member since
Jan 9
Recent submissions
Comment · 5 years ago
I am glad you like it. Hyperband relies on early stopping of the model. So it is something you would do inside the objective function by yourself. I have plans to add some helper functions for early stopping in the future.
Comment · 5 years ago
Thank you very much :-) yes you can put anything you want into the search space. Even pandas dataframes, numpy arrays or classes. Here is an example: https://github.com/SimonBlanke/Hyperactive/blob/master/examp...
Comment · 5 years ago
Check out the Neural Architecture Search Tutorial here: https://nbviewer.jupyter.org/github/SimonBlanke/hyperactive-...
Neural Architecture Search is just one of many optimization applications you can work on with Hyperactive. Check out the examples in the official github repository: https://github.com/SimonBlanke/Hyperactive/tree/master/examp...
Comment · 5 years ago
Yes it is quite easy to switch algorithms via the "gpr" parameter. You just have to write a wrapper class. I am currently working on a repository that discusses how to do that in detail: https://github.com/SimonBlanke/surrogate-models
I will upload some examples (GPy and Gpflow) within the next 24 hours.
I think those wrapper-classes will already answer some of your questions. If you have additional or unanswered questions you can open an issue in the Grad-Free-Opt repository.
Comment · 5 years ago
Yes Hill Climbing + some of its variants are featured in this package.
Comment · 5 years ago
Unfortunately NLopt cannot be included into Gradient-Free-Optimizers. But i would like to implement multiple algorithms that are also preset in NLopt.
I think it will help my understanding of optimization algorithms if i implement them myself. I currently have my eyes on the Downhill-simplex, Direct and Powell's Method. But they are quite different from what i have already implemented, so this could take some time.
Comment · 5 years ago
Your parallel computing approach sounds intriguing! Could you provide an example script? I would like to look into this. If you like you could open an issue as a feature request and provide a code snipped there.
Comment · 5 years ago
I thought about a table in the readme that shows some kind of metric for each optimizer that describes its performance. I will look into that.
Comment · 5 years ago
I will look into this algorithm. Thanks for the suggestion. I have some basic explanations of the optimization techniques and their parameters in a separate repository: https://github.com/SimonBlanke/optimization-tutorial
But there is still a lot of work to be done.
Comment · 5 years ago
Two very interesting questions! I should work soon on a comparison of Hyperactive and GFO to other packages. If some important features are missing, maybe i could add them.
I will also look into Dlib. If you like you can open an official issue in my repository. We could discuss why it is useful and if it should be implemented.
Comment · 5 years ago
Gradient-Free-Optimizers has a type of genetic algorithm: Evolution Strategy
Comment · 5 years ago
Evolution strategy is included. The "Covariance matrix adaptation" is for making this algorithm work for continuous search spaces. But gradient-free-optimizers has discrete search space.