This is certainly annoying for new players.
You are told to talk to Ailis in the downtown, you pick up your caravan destination, and then you are told to talk to caravan leader at the entrance. Which is entirely false, as the person you need to talk to is in downtown.
Please fix this quest.