A result by Tange from 2015 gave bases for the spaces of highest weight vectors for the action of GL_r×GL_s on k[Mat_{rs}^m] over a field of characteristic zero, and in arbitrary characteristic for certain weights; here, we generalise this to give bases for the spaces of highest weight vectors in k[Mat_{rs}^m] of any given weight in arbitrary characteristic. The motivation for this is to apply the technique of transmutation to describe the highest weight vectors for the conjugation action of GL_n on k[Mat_n]. Then, we use similar methods but in characteristic zero to describe finite spanning sets for the spaces of highest weight vectors for a certain polynomial action of GL_r on k[Mat_r^l] (derived from the GL_r-action on Mat_r given by g·A=gAg^T), and apply this to the conjugation action of the symplectic group Sp_n on k[sp_n].