It is mentioned here (SB 3.1.2) that the Lord identified the house with His own Self. Thus the house of the Pandavas was as good as Vrndavana, and Vidura should not have given up that place of transcendental bliss