Dear Guilherme,
Does your model account for structural pretwist, as FAST does? As discussed in the following forum topic: Coupled blade modes in FAST, because the blade flapwise stiffness is typically quite a bit less than the edgwise stiffness, the influence of the edgwise bending on the flapwise tip deflection (due to pretwist) will be much less than the influence of flapwise bending on the edgewise tip deflection. Perhaps this is why the results between FAST and your model match for out-of-plane deflection, but not for in-plane deflection.
Best regards,