At Netflix, we have considerable use of the autoscaler with our custom scheduler. Is there any reason you can't autoscale based on queue depth, or time of items in queue?
As far as your Docker problems go, have you tried to upgrade to 4.9.34+? We found the issue as well, and upgraded kernels, and haven't looked back.
As far as your Docker problems go, have you tried to upgrade to 4.9.34+? We found the issue as well, and upgraded kernels, and haven't looked back.