Comment by mattfrommars

Are you using any kind of job scheduler to do the job for you in the background? Are you using Apache Arrow for job scheduling since you mentioned you need to apply to jobs behind the scene and spread out across day and night.