Disconnecting / removing peers form the DHT and peerstore

In my case, I want to only allow peers to connect which are known to me.

Therefore, I am adding a custom handshake, a bit following Implement custom handshake - #7 by MOHANKUMAR-IT

However, if my custom handshake fails, the DHT module still reports being connected to the peer. if I check via host.Peerstore().Peers().

My disconnect code in case of a failed handshake does

	host.Network().ClosePeer(p)
	host.Network().Peerstore().RemovePeer(p)
	dht.RoutingTable().RemovePeer(p)

But still via the PeerStore the peer is there…

Am I thinking about this in a wrong way? I understand that the DHT is managing the mesh underneath, but I really want only my valid peers to connect…