Playing around and testing out Magic, I've had these squares show up when I reopen files and I can't get them to go away

Questions about magic should be addressed to #magic or they are less likely to be answered. The "square" in question is a label that is in a subcell. You can't get rid of it unless you are editing the subcell that it is in, which you can do by selecting an instance of the subcell (

i

key) and then editing that instance (

e

key).