Hello,
Thank you for posting in Q&A forum.
Yes, the session can be kept and refreshed even the password is changed.
From network aspect, once you detect any suspicious IP address you can prevent the session by blocking the IP address. Otherwise, you need to check if there's any securify measures for your application (e.g. Microsoft Defender for office 365 can safeguard your organization against malicious).
To help other customers who may be facing the same issue, please don't forget to vote if the reply is helpful.
Best regards,
Jill Zhou