In order to avoid limiting users to specific applications you can leverage Custom Policies in tandem with features such as Restrictions (for local accounts) so that only specific issuerUserId values are allowed to be input or used (EG: users from selected domains trough a regular expression) or claims transformations such as SetClaimsIfRegexMatch and AssertBooleanClaimIsEqualToValue to output the result of matching the issuerUserId with the same regular expression used before and raise an error according to the result.
SSO sessions can be scoped to application, policy or even disabled.