Currently, when inviting a user to a group, the invite link is valid for 30 days. There is no way to change that time period.
Which means that, if you remove a user from a group within that time frame, they are able to rejoin.
We use groups for beta testing with requirements. If people no long match the requirements, we need to be able to boot people from a group without them being able to rejoin.