The non-equilibrium steady states of integrable models are believed to be described by the Generalized Gibbs Ensemble (GGE), which involves all local and quasi-local conserved charges of the model. In this work we investigate integrable lattice models solvable by the nested Bethe Ansatz, with group symmetry $SU(N)$, $N\ge 3$. In these models the Bethe Ansatz involves various types of Bethe rapidities corresponding to the "nesting" procedure, describing the internal degrees of freedom for the excitations. We show that a complete set of charges for the GGE can be obtained from the known fusion hierarchy of transfer matrices. The resulting charges are quasi-local in a certain regime in rapidity space, and they completely fix the rapidity distributions of each string type from each nesting level.