@Tim Edwards Thanks for the quick reply.
(1) I'm not sure I understand your point here. Or maybe I do understand if you are referring to the headers J3 & J5 which are indeed shorted by default. I'm talking about J8 & J9 where you can switch the Nucleo hat between MAIN and HAT mode. They are not shorted, I've checked both with a multimeter and in the kicad schematic. Also if I don't install the jumpers the chip isn't getting any power. And the docs at
https://github.com/efabless/caravel_board/blob/main/firmware_vex/nucleo/README.md are indeed asking me to install the jumpers. My only nitpick is that it's not telling me where to take the jumpers from and there were no extra jumpers in the box. If you say "we expect anyone doing electronics to have a box of jumpers", that's fine. If you say "take off the jumpers from <whichever locations> and put them at J8 & J9 in the HAT position", that's also fine.
(2) There are also low quality cables that have a chance to temporarily disconnect if you touch them the wrong way. They used to be less of a concern, but these days higher-end electronics usually come with usb-c cables and lower-end ones come with micro-usb. Which means if I grab a random micro-usb data cable from my drawer, it has a pretty high chance of being a low quality one. (It's not a real issue for me as I could borrow the cable from my other MPW-3 project, but I felt that I shouldn't need to.)
(3) Ok, thanks.
(4) My guess would be that the lack of "pass" means untested. I was wondering whether the boards which did have the "pass" marking failed because something is different between our setups (maybe I'm doing something wrong) or because it was something entirely different that was tested and passed.