There are many ways of forming groups. Below are a sample and you can mix elements from each too, depending on your needs.
Heterogeneous groups: In this form of grouping, the groups are formed specifically to ensure that they have diversity within them. The diversity considerations could be ethnic, gender, geography, math/verbal skills, learning styles, quantitative reasoning abilities, etc.
The point of heterogeneous groupings is to ensure that people get to know and learn to respect the contributions of others who are different from them.
Arranging such groups requires the instructor to have some prior knowledge of the background of the students.
Some researchers recommend that heterogeneity also include 'high' and 'low' ability students. They argue that this tends to improve the performance of both the 'weaker' students (because they can draw upon the knowledge of the 'stronger' ones), and the 'stronger' ones (because having to teach the weaker students improves the skills of the strong students).
While I agree with the rationale itself, I do not use 'ability' as a criterion because I am not sure of its validity or my ability to judge it. I believe that prejudging a student’s 'abilities' can be difficult, misleading, and self-fulfilling.
Random groups: Here the groups are formed on some random basis using birthdays, playing cards, numbering, etc. This avoids the risk of the instructor being perceived as being partial or unfair.
The problem is that you might accidentally generate groups that have undesirable properties, such as all being segregated or with strong personality conflicts or all people who are struggling with the academic demands.
I tend to use random groupings only if I have no prior knowledge of the group, for tasks that do not require a lot of prior knowledge, or for very short-lived tasks where personality conflicts are not likely to develop into serious crises.
Self-selected groups: In this method, the groups are asked to form themselves. This has the advantage that the groups tend to start off with good feelings towards each other (though there is no guarantee that this will last). One problem is that you might end up with groups of members who were already friends. The qualities that tend to make for good friendships are not the same as those that go towards good group work, and the danger is that the groups might be ineffective and that friendships might fall apart.
Also self-selected groups tend to be homogeneous in terms of gender, ethnicity, and personality styles (such as being task-oriented versus non-task oriented), and this works against one of the goals of active learning, which is to learn to be able to work with diverse peoples.
And finally, self-selection frequently results in hurt feelings for the students who are picked last.
For all these reasons, self-selection is the least desirable mode of group formation. I suspect that the main reason it is used is that then, if a group fails to function well, the instructor can avoid the blame for its dysfunction.
Mixed mode groups: In this method, the instructor has final say in the selection of the groups but students have input into the process. Prior to the group formation process, students can be asked to confidentially list (say) three persons they would like to have in their groups and fewer (one or two) persons they don’t want to work with. The instructor then tries to form heterogeneous groups in which at least one of the persons they liked is included, and the persons they disliked are excluded.
This method can also be used to identify students who feel they do not belong and are unrequested, and place them in a supportive group where they might feel more valued and welcome.
Functional mode: In this mode, group members might be selected on desired criteria. At CWRU, which is a residential university where nearly all the students live on campus, I form groups in which all the students live in one dormitory or adjacent domitories to make it easier for them to meet outside of class. Commuter students are put in separate groups.
Another way of forming groups is to mix expertise (computer, writing, graphics, statistics, etc.) so that each group has the necessary complement of skills to do the project.
Learning styles or personality types can also be used to form groups, so that each group has a diverse mix,