dmillard

Clearly not - it's quite funny (or at least I thought so).

Anyway, the author is a well known writer in the robotics world.

Great questions and a few answers to various parts of your question that you've mostly identified yourself I think:

- Golf clubs specifically (and most sporting equipment) actually goes down a different pipeline in most airports, since generally this stuff doesn't behave well on conveyor belts.

- Data driven approaches can tell you a lot just from visual information, usually about deformability of objects, but also about expected centers of mass, etc

- Part of the reason we're using single big robots is because you can use heavy duty end-effectors - grasp all the way around the object with a grasp that's predicted to be robust to these kinds of perturbances, and then use quick feedback to safely execute a plan to place it

You're absolutely correct though, that there's a long tail of things coming through and that some objects are very, very difficult. Our problem formulation then becomes identifying confidence in graspability, and deciding explicitly that we shouldn't attempt to grasp some object and should instead flag them for human handling.

Not a pest at all and I've long been frustrated with ROS - our early demos were actually just a single C++ binary with multiple threads running for perception, control, and visualization, and I byte-packed robot control messages in our own software to avoid using ROS.

Unfortunately this breaks down in a few ways that you're probably familiar with, given that you asked this question:

- A crash in (e.g.) a third party sensor driver can bring down your whole binary, any signal catching here is awkward and you end up wanting process isolation

- Perception is, for better, or worse, easiest to prototype and try off the shelf in Python / Pytorch, so you either end up with pybind11 and driving things in Python, ONNX which is IME brittle for some of the crazier Pytorch modules, or message serialization and process isolation.

ROS/ROS2 does _way_ too much in my opinion - why does it have a build system and a ton of packages? This plus pinning OS versions are huge pain points. Unfortunately I also think many community-contributed ROS/2 packages are fairly low code-quality, with notable exceptions. Overall, I'd prefer to have ROS be a pubsub library with a few extra tools for logging and visualization.

In the end, we're currently using ROS2 for the reasons listed above and for easy prototyping, but I'd like to move to something more like FPrime, Basis, Cerulion, or Copper in the near future. I really want to grow something in-house with Zenoh or IceOryx2, but don't want to waste a lot of time on middleware, since I don't think it's what's kept the problem from being solved.

(At the end of this post I now see you're working on Basis, I apologize that I'm over-explaining to you. I'd love to chat about Basis if you have some time in the next few days!)

There have been some solid attempts in this space before - many projects take on the whole baggage system design and end up very very complex and often over budget. We're focusing on introducing tech that plays well in a larger system, particularly in "brownfield" existing processes - our bet is that recent advances in robot autonomy give us ability to handle items that weren't possible before, and therefore our units can be introduced in a more flexible way.

Thanks for the feedback!

John B was obviously aware from previous experience what a manual and injury prone process this was, but I've also been really surprised as I've dived deeper into airport operations myself.

Bagroom is definitely what we're targeting first - being indoors (usually) is a huge plus, and lets us focus on the manipulation part of the problem without going fully mobile yet.

That said, we're definitely targeting tarmac/ramp operations, particularly between a TUG/PowerStow and narrow-body bag carts. Inside the bin is much trickier but we agree it's the least ergonomic part of the job, you just can't move a massive industrial arm in and out of a plane very easily. We have it on our longer-term roadmap, though, and intend to leverage the baggage dynamics data we collect everywhere else to give us a head start on the packing and manipulation problems there, just with a different mechanism.

Cargo packing is a huge area of interest for us! Particularly around optimizing weight distribution in loaded planes, or just optimizing packing efficiency in general.

Excellent post! Curious if WebRTC can be adapted for 3d sensor data and would love to chat more about it - I'll send an email!

We're really focused on health and safety aspects of this job - in a repetitive stress sense, these jobs are much more dangerous than many people imagine they would be and people end up with lifelong injuries.

Generally, regulators seem to be moving in this direction as well. The EU has introduced new regulations on the total amount of weight someone can move in a shift, and the Dutch government has mandated that baggage handling move away from manual processes like this in the near future.

Suction has gotten us pretty far at the prototype level but definitely isn't enough - we're testing out some new gripper designs that use suction as a broader part of an overall grasping system.

For these videos we have lidars and two Intel Realsense depth cameras mounted to the safety cage and on a wall. We're working on moving as many sensor on-robot as possible in the near future to aid with deployability.

The cup has taken us very far, which we're excited about, but it's definitely not enough - we're currently testing a multimodal claw-ish + suction gripper, which we've had good results with so far but aren't ready to unveil.

The teleop data is really useful for training data indeed, and lets us collect data on current failure points (e.g. with suction, just how far can we tilt this fabric bag before it peels away, etc). We're not going full behavior-cloned end-to-end for a lot of reasons (sample complexity, safety, adaptability, etc), but we do a lot of learning in specific parts of the system (particularly around grasping and placement).

The robot is indeed beefy, as many robots rated for 50kg applications are (check them out online). We've accidentally stress tested this unit way beyond 50kg without a hiccup, so we're very interested in figuring out what the right-size unit is for our application. There are a few other great aspects to this unit - it's a 7-DOF arm + 1 more DOF for the linear rail, so we have two extra degrees of freedom to play with for collision avoidance during planning.

Teleop and monitoring are systems that we've built ourselves and are pretty happy with. Since we use MuJoCo for simulation/visualization and some kinematics subroutines, to visualize, I just keep the MuJoCo GL context open after rendering and then throw all of our sensor data into it - it's very performant and low latency!

We've since introduced a message-bus layer that makes it possible to do it all over the internet etc, but adds the associated serialization and transport latency.

Operational changes for airlines are quite tricky - one of our bets is that most of the value for customers here is in handling "brownfield" deployments where you drop into an existing process, and that intelligence (or at least, good perception and reactive planning) really unlocks this ability from the robotics side.

For widebody planes, bags are already loaded into Unit Load Devices (ULDs), which are large semi-truncated boxes that get loaded directly onto aircraft. Narrow body planes don't use these (apparently) because they impact turn-time and decrease the amount of time a plane can be in the air, and also impacts how quickly bags come out, since it adds an extra step to unloading.

Many airport conveyance systems also load each bag into a bin, but those bins aren't loaded into the airplane because they belong to the airport and waste space/weight.

The best case for us would be a customer process change where everyone loads their luggage into perfectly regular and very sturdy hardshells, but this one's probably out of our hands.