A graduate student takes classes, researches a (usually quite narrow) topic area with extreme depth, writes papers for class and for publication, teaches, and works with mentors towards an academic goal. An un-graduate student does all the same things, but without the structure of a university or any other single-point-of-contact authority guiding the academic work.
A graduate student has a well-defined path toward a degree and/or certification. An ungraduate student has no defined path except the one that they lay out for themselves, and there is no institutional degree at the end of the process. There may or may not be some certification available via testing for the ungraduate at the end of a period of study.
Ungraduate studies are an extension of unschooling. It is a form of lifelong learning.