Silly mistake on my end, of course we want to build on both arches.

Fixes: 9eb4d27c ("packit: only build F41+")

Signed-off-by: Paul Holzinger <pholzing@redhat.com>