It wasn't as much the idea of aliens as it was the execution. The Ark was memorable because it literally melted faces and made for one hell of a climax. The Temple of Doom (the temple itself) was memorable for about 243 different reasons - Mola Ram pulling hearts out, the eating of delicious chilled-a monkey brains, the crazy mine cart chase.
Last Crusade's Canyon of the Crescent Moon, also insanely memorable. The traps, the "you chose wisely" crusade knight, the doomed golden chalice, the face melting revisited, and Jesus' brown cup of austerity. Plus Henry Sr. laying there dying the whole time. ONLY THE PENITENT MAN WILL PASS.
Indy 4 didn't have any of that shit. Outside of Harrison Ford, who doesn't even have to try to kick ass in the Indiana Jones role, the movie was entirely without a soul.