I've only just started experimenting with changing clock period

Even though you use 10ns to synthetize this value could change due to PVT, I don't know if openlane uses typical case for timing analysis. If it uses worst case you can use 20ns an get a higher frequency on silicon.

Will simply increasing the clock period work for caravel? Won't that affect the rest of the chip or is PLL going to deal with that?

my understanding: all the rest of the caravel chip has already been hardened. AFAIK top frequency is ~50M. So when you are hardening just the bit inside, it depends what your target is. If you're going to be running everything at 10M for example then you only need to set the clock period to match that.