Why didn’t the Japanese Invade the Soviet Union and Help Finish them off?

Even though they shared a heavily-militarized border, the Soviet Union and Imperial Japan didn’t fight until the very end of the war. Why?