I soldered a 15pF capacitor between the clock line and ground and it seems to have fixed my flashing problem. Flashing has gone from a very sporadic thing to working most of the time. The capacitance value is going to be a balance between fixing this issue and causing timing issues with the processor accessing the SPI flash during operation.