I don't know why it took so long for me to figure this out but I can just change the discover port. On the device I currently test with things worked so either Android 10 is different or it might be fixed in 1.6.10 prerelease that I am using. I will try on the ONN tomorrow.
Testing on various Android 10 device confirms the problem with no local devices, it relates to Android 10 opening uidroid twice. I've fixed it for the next release. Android 11 and 12 are OK