markrwilliams
Joined 13 years ago
Karma
22
Submissions
10
Member since
May 2
Recent submissions
Comment · 1 month ago
Comment · 1 year ago
I'm an AI skeptic but the chatbot sounds interesting. What kind of context do you provide it?
Comment · 1 year ago
Nope! There are evented eBPF map types that userspace processes can watch with epoll(2), e.g. https://docs.ebpf.io/linux/map-type/BPF_MAP_TYPE_RINGBUF/#ep...
Comment · 8 years ago
PEP 412 makes __dict__s more memory efficient than they were before, but not more efficient than no __dict__, which is the point of __slots__. The following program demonstrates the difference. Note that it lowers the available address space to 1GB so that memory exhaustion occurs sooner, and thus only works on UNIX-like systems that provide the resource module.
import resource
import sys
class WithoutSlots:
def __init__(self, a, b):
self.a = a
self.b = b
class WithSlots:
__slots__ = ('a', 'b')
def __init__(self, a, b):
self.a = a
self.b = b
resource.setrlimit(resource.RLIMIT_AS, (1024 ** 3, 1024 ** 3))
cls = WithSlots if sys.argv[1:] == ['slots'] else WithoutSlots
count, instances = 0, []
while True:
try:
instances.append(cls(1, 2))
except MemoryError:
break
count = len(instances)
del instances
print(cls, count)
$ python3.6 /tmp/slots.py
<class '__main__.WithoutSlots'> 5830382
$ python3.6 /tmp/slots.py slots
<class '__main__.WithSlots'> 16081964
https://morepypy.blogspot.com/2010/11/efficiently-implementi...
Comment · 11 years ago
If you get a file descriptor that refers to a child process upon its creation, then that file descriptor should behave like other file descriptors.
That means you ought to be able to transfer it to other processes via file descriptor passing (the SCM_RIGHTS ancillary message; see man unix).
The identity of a process would thus be local to its parent or to a process with which the parent has agreed to share that identity. Not only does this avoid race conditions, it also enables a completely unrelated process to reap a child which can be terrifically useful.
This is exactly the approach the Capsicum sandboxing framework (mentioned elsewhere) is taking. The goal there, though, is to eliminate globally shared identifiers as much as possible -- which makes sense for sandboxing!
Comment · 11 years ago
> The "self-pipe trick" is ugly, involves a lot of unnecessary overhead, and runs the risk of deadlocking if you receive enough signals to fill the pipe buffer before you read them back
The unfortunate terseness of the original "self-pipe trick" description makes the solution to this difficult to see. As far as I've figured out there are two things to notice:
1) You're supposed to set the pipe to be non-blocking. Presumably you also then don't check the return code of the write(2) call in the signal handler. While this solves the case of a signal handler blocking forever, it does mean you might have dropped writes that correspond to signal receptions. That leads us to:
2) The self-pipe trick specifically calls out handling SIGCHLD (probably because it's one signal that you don't want to ignore!) But given the chances of dropping a byte as described in 1) and the fact that SIGCHLD and fork are explicitly called out, I can only assume that the lesson here is: only have one pipe per signal you intend to handle. Since multiple signals sent to a process may result in a single signal being delivered, your real signal handling code (the stuff that's watching the other end of the pipe) already has to deal with this situation.
As for Capsicum, I can't wait til they implement pdwait(2)! Until then, at least pdfork(2) ensures that the parent process' death kills the child process...
Comment · 11 years ago
It's tendentious to describe gevent as deprecated.
Like Twisted, there are many projects that continue to choose to use gevent, and consequently (for either of those libraries) Python 2. They solve real problems, as does the codebase we extracted SuPPort from, and we hope to help these projects rather than shun them.
Additionally, asyncio doesn't provide the same user API that gevent does: automagic task switching. Whether or not you like that API is a separate issue.
Finally, even if a person has no interest in using SuPPort, we hope that it might provide her insights into problems she'll likely encounter when using asyncio (or Tornado or Twisted or eventlet or any evented system.)
Comment · 12 years ago
You're underestimating the amount of effort put into simplifying and clarifying implementations, APIs, and documentation.
This doesn't imply that preceding implementers are unskilled. Part of this improvement is hindsight, but a not-insignificant part is Python's flexibility.
I can vouch for the apparently unbelievable productivity gains but I'm just not sure how to prove them!
Edit: As for not sharing these gains -- that's part of the point of this post to the official PayPal engineering blog :)
Comment · 12 years ago
Disclaimer: I work with Mahmoud.
Presumably "ASF" doesn't mean anything to you. That's because this isn't an image macro, but rather a slide from an internal presentation that compared PayPal's Python API for a custom serialization format to PayPal's C++ API for the same.
Consequently it's important to keep in mind that advances in C++ aren't uniformly available and expertise in it is hard to acquire. That's why the image appears in a section describing the productivity wins available in Python. The point is that Python allows us to develop something that presents a clearer interface to a powerful and performant implementation, and allows our users to worry less about blowing their feet off.
Comment · 12 years ago
Pitch-perfect satire. The slow build to heavy-handed propaganda is especially great.
"On Linux 6.9 and newer (6.10 on arm64, where JIT support for the arena landed later), the window is backed by bpf_arena... On kernels without a usable arena, the same four gigabytes are assembled from 4-MiB pieces... [in] separate global-data maps."
These maps are of type BPF_MAP_TYPE_ARRAY.
See https://lwn.net/Articles/961941/ for a discussion of `bpf_arena`.