@Ananya Sarkar , I have tested this in my lab where I observed that Public IP added to an Azure FW using PowerShell is missing on the portal view but is been tagged to the firewall in the backend. See this screenshot,
Explanation: azFwPublicIp1 is the IP configuration I have added to firewall using PowerShell. But as I can't see this on the firewall, I decided to tag from the portal. Then I faced this error saying it is already tagged in the backend.
PIP added with PowerShell Commands:
Azure Portal: It reflects in the IP configuration list and working as expected if I create it from the portal,
(My Public IP name = second-ip) --> for your reference
So, it looks like a kind of bug. We will report this to the respective product team for a quicker resolution. Meanwhile, please try using other methods of deployment like Portal or CLI.
----------
Please do not forget to "Accept the answer" wherever the information provided helps you to help others in the community.