Dear Jun,
I’m not sure I fully understand your question, but the structural damping specified for the blades and tower in ElastoDyn has similarities to, but is not identical to, Rayleigh damping. Instead, blade and tower damping in ElastoDyn is only based on the stiffness-proportional term and is specified independently for each mode. This is unlike Rayleigh damping, which as you point is, has both mass- and stiffness-proportional terms and is not mode-specific.
A similar topic was discussed in the following forum topic: Natural frequency and damping ratio calculation - #10 by Jason.Jonkman.
Best regards,