The relationship between n_permutations and exact test in permutation test

[mmagnuski (Mikolaj Magnuski)]

Yes, if the number of samples is small you don’t have to approximate the permutation distribution via Monte Carlo (random draws), bacause you can check every possible permutation. As you say, doing it randomly you risk repeating the draws.