dante9999
Joined 13 years ago
Karma
178
Submissions
39
Member since
Aug 25
Recent submissions
Comment · 1 year ago
US politics is broken, but most AI regulations are poorly designed. Look at EU AI policy. It is not addressing any real problems and is mostly just additional paperwork.
Comment · 10 years ago
sounds like you should ask for your money back, remember OP stands by everything he creates and if you dont like it he promises money back no questions asked
Comment · 10 years ago
> I know a lot of people that wrote books and got something out of it.
did they self-publish or went with some known publisher? what kind of money can you make on ebooks?
Comment · 10 years ago
I dont think he means to deprecate enjoying life and spending time with friends and family. It's more about good healthy life habits - nutrition, exercise, sleep. When you think about it all these good habits make you happier. He certainly does not advocate workaholism. At least I dont read it this way.
Comment · 10 years ago
> Does that all make sense?
yeah it does make sense.
> in a real-world deployment you couldn't limit connections with client-side code
yeah that's a very good point. But in a real world scenario handling this would not be that easy. Limiting number of available connections on the server side is not a trivial task to implement. Setting your server to avoid failures and simply return either 503 service unavailable to some clients or 429 (too many conns) to others would probably require quite a lot of coding. It's also not very clear to me how this would be implemented, how do people implement things like this? Just putting some check for number of open files before line that opens file and setting response code to 500 and 429 before opening file? This would only stop server from opening to many "html" files, but would not stop server from getting flooded with connections. Is my aiohttp app even the right place to add checks like this? Wouldn't it be better to use haproxy or nginx or some other load balancing service in front of aiohttp app and let it handle too much traffic?
Other thing that comes to my mind (need to check this later) is that perhaps some partial "handling" of cases like this could/should be implemented in aiohttp library. I'm not sure how it behaves now, but maybe it should simply fail to open file, return 500 to the client, and print noisy traceback about open files to my logs? I didnt see this behavior when doing my tests, so either it didn't occur, is not implemented in aiohttp, or it occurrred and I somehow missed that. From my experience with Twisted I know that this is how Twisted resources behave, if you have some unhandled exceptions twisted just returns 500 to client and show traceback in logs.
Comment · 10 years ago
Thanks for feedback.
> You would probably handle more requests if you changed that -- I would do the file access in a run_in_executor with a max executor workers of 1000.
This is really good point. I'm going to check this and edit post adding this information there.
> Also, the placement of your semaphore acquisition doesn't make any sense to me. I would create a dedicated coroutine like this:
looking into my semaphore code next day after writing it I do wonder if I'm using it correctly. I assumed it works correctly because it fixed my "too many open files" exception, so it seems to mean that I'm no longer exceeding 1024 open files limits. Can you clarify why you think my use of semaphore does not make sense and why your suggestion is better? What is the benefit of dedicated coroutine?
> That being said, it also doesn't make any sense to me to have the semaphore in the client code, since the error is in the server code.
I admit that I focused more on my client than server. One thing that worries me about my test server is that it does not print any exceptions. Either it does not fail at all, which seems unlikely, or it fails silently, which is more likely and is bad. So I need to check my server code to see what exactly happens there.
> it also doesn't make any sense to me to have the semaphore in the client code, since the error is in the server code.
main reason for semaphore in client code is that it should stop client from making over 1k connections at a time. My logic here is that if client wont make 1k connections at a time - server wont receive 1k connections at a time and thus there will be no problem of too many open files on server (it won't have to send more than 1k responses). However I see that this logic may not be totally correct, other comment points out that it's possible for sockets to "hang around" after closing: https://news.ycombinator.com/item?id=11557672 so I need to review that and edit post.
> https://docs.python.org/3/library/asyncio-dev.html#debug-mod...
this looks really great, will look into this thanks.
Comment · 10 years ago
I wonder if you can reliably classify jobs into "nonroutine" and "routine". There is element of routine in every work, and I'm pretty sure that even most boring and repetitive job can be done better with some degree of creativity. It would be really interesting to read more about reasoning behind classification presented in this article. I mean can you seriously say there is no "routine" in programming or management?