How do I do the equivalent of “save as” to make a new version of a whole project?
Can I just duplicate the project folder and rename it or will that cause lower level conflicts?
Theoretically it’s not good to copy the whole project directory since it also copies the project’s UUID which is intended to uniquely identify a project (it also appears in generated Gerber files). But in practice it doesn’t really matter since that UUID is not used for anything important.
The most important thing is: Even if you copy a project directory, it will be a 100% independent clone of the project, with no relation at all between the old and the new project. So feel free to do it this way 
Are there also other UUIDs in the projct file that will be copied?
Can there be any problems if I have new and old versions of a project open at the same time?
Since you say it is OK I will try it. I think I will rename the board and schem files too otherwise it will get confusing.