I would suggest we just remove the inner set of holes and leave the pads for the pogo pins. If someone wants to use pin headers, they can connect to the castellated holes.