OOC:
Just to clarify, Panzer-
The entrance to the fortress is basically a magic portal that moves you from the cave up near the surface, down to the underground bunker complex which is under something like a mile of rock. It
simulates traveling down a spiral ramp. The ramp is sort of like a holding pattern (load screen?

) that the fortress holds you in while determining whether or not to let you in.
The more you think and act like someone the fortress wants inside, the faster your journey is. If you're thinking "argh you bastard deVerett, I'm going to come in there and kill you," the fortress would probably keep you spiraling around in a holding pattern until you died of thirst and/or boredom.* If you're thinking "I will go down and render medical assistance to the poor unfortunate people in the fortress," like Rohal did, you get in pretty fast.
So the question is, how strong was Alfred's sense that he ought to be in the fortress? The stronger it is, the less distance he travels. And how closely do Alfred's motivations line up with those the fortress wants to see in people coming into it? That has an effect too.
Larric's problem, for example, is that a large fraction of his sense that he ought to be down there is curiosity about the fortress. And if the fortress is running a TSA-style scan of your thoughts to decide whether or not it lets you in, "I want to learn the fortress's secrets" is one of those things you do NOT want to say while waiting in line. It gets you the extra-double-plus-groping and interrogation, or in this case a climb that makes you feel like you just hiked down a mountain.**
So ask yourself- how long was the trip for Alfred? Is he going to be willing to make the (possibly very long) climb up to the surface bearing people on his back, before he accomplishes any of his other duties? How would his knowledge of what is entailed in all this affect what he says and does?
____________
*It might have more... direct ways of making its displeasure known to someone who wants to attack the people it's protecting, but let's not think about those right now.
**Come to think of it, that may be another way for Verone to part ways with the party- for him the climb is like ten miles down, or the fortress just plain spits him back out since it detects tons of curiosity but no hostile intent, or something like that.