Match and extract if found

[Calli]

For instance I have a list of IP address in one file IPfile1.txt

192.168.0.1
192.168.0.2
192.168.0.3
2001:4800:7819:104:be76:4eff:fe04:5819
192.168.0.4
192.168.0.5
192.168.0.6

And I have another file IP2.txt

192.168.0.799
192.168.0.900
192.168.0.3
2001:4800:7819:104:be76:4eff:fe04:5819
192.168.0.1000
192.168.0.5
192.168.0.83

What I want to achieve is SCAN the IPfile1.txt file using IP2.txt and if the ip address from IP2.txt found in IPfile1.txt than remove that IP and output the ip which is not in IPfile1.txt which in this case is
192.168.0.799
192.168.0.900
2001:4800:7819:104:be76:4eff:fe04:5809
192.168.0.1000
192.168.0.83

And skip anything which isn't an ip address which in this case is 2001:4800:7819:104:be76:4eff:fe04:5809

Can anyone help me with this program