A cursory glance at other CAD apps shows STEP is everywhere, and STL/AMF are fairly common.
The new version of STEP can contain colors it seems, so if STEP alone is good enough for a basic fallback, you could have STEP required, and allow arbitrarily many other files as well.
It seems like VRML is the one of the only things that really has an accepted way to put materials with specular reflection and the like into a file, aside from going with an app-specific format.
One could define a standard set of STEP material names, and an exporter that knows what material to apply to a GoldPlatedPin or some Solder or some BlackChipEpoxy. That kind of abstraction gives flexibility and prevents being forever tied to one model of how rendering should work.
OBJ seems to explicitly be designed for that kind of workflow, with named materials and a separate MTL thing, and it’s pretty bog standard. The MTL files have transparency and specular reflection and such.
You could just have a basic flat color+openGL viewer integrated, and leave HQ rendering to a Blender exporter with a library of shaders for each material.
This would also save parts authors from having to worry about presentation too much. They just type a material name, and LibrePCB handles making it look good.
Another pretty big advantage is that different people’s models will have the same color chips and pins and such in the render, preventing an annoying cacophony unless a part really does have a specific color.