To love God is to have faith in Him and to obey Him. Obedience here is key.
During the period Jesus describes, false prophets will appear and direct people away from God. Those who allow themselves to be swayed will stop recognizing His authority, following His instructions, and conforming to Him. It’s this lack of obedience that will result in relationships with God to suffer and the love for Him to grow cold.