Dear Magnus,
You can get the full-system mode shapes through the linearization functionality of OpenFAST, followed by eigenanalysis of the state matrix A. The resulting eigenvectors contain the full-system mode shapes, but these are not mass normalized. In fact, it is not really possible to get mass normalized mode shapes from OpenFAST when a combination of SubDyn and ElastoDyn are enabled to model the structure because the full system mass matrix is never formed in that case.
It is challenging to specify the damping of the support structure in OpenFAST given the combination of Guyan and Craig-Bampton modes in SubDyn and the tower mode shapes in ElastoDyn. (The Guyan modes in SubDyn involve motion of the transition piece, the Craig-Bampton modes in SubDyn assume a fixed transition piece, and the tower modes in ElastoDyn are influenced by the compliant substructure.) This challenge of specifying support structure damping has been discussed briefly in the following forum topic: Subdyn with Full FE option - #5 by George.Chan.
Modeling the full support structure in SubDyn and neglecting the tower model in SubDyn simplifies the process a bit, but does not fully resolve the issue, because in that case, the support structure is still represented as a combination of Guyan and Craig-Bampton modes. (In that case, the Guyan modes involve motions of the tower tower and the Craig-Bampton modes involve a constrained tower top.)
SubDyn can be used standalone (without ElastoDyn, uncoupled from OpenFAST). In that case, you can represent the RNA as a lumped mass and inertia through the “Joint Additional Concentrated Masses” option in SubDyn.
Best regards,