Because Ricardo Montalbán had played Khan in The Original Series, Star Trek II director Nicholas Meyer was not involved in casting him — although he certainly had no complaints.
“There’s an actor!” he told Enterprise Incidents 16 (April 1984, My Star Trek Scrapbook has the full interview). “He’s living proof that good manners don’t have to be bought.”
The first scene they shot was also the first in which Khan appears in the movie. “It was six minutes of monologue with 23 camera moves back and forth.” Montalbán knew his lines cold and got it “perfectly”.
Yet that’s not the version that ended up in the movie.
Meyer remembered telling Montalbán, “I think you are much more powerful and much more dangerous if you’re very quiet, if the audience doesn’t know what you’re capable of.”
Laurence Olivier once said, “Never show an audience your top, because if you do they’ll know you have no place to go. It’s better to keep it locked down.”
Montalbán understood instantly. “It just clicked with him.”
For Carol Marcus, Meyer wanted an actress who looked beautiful and smart; “a woman who was attractive enough that you could see why Kirk would fall for her, and at the same time somebody who could keep up with him,” he told Star Trek: The Magazine 3, #5 (September 2002).
I loved Bibi Besch; I became very close with her, and I used her again in The Day After. She’s no longer alive and I bitterly regret it; she was a lovely human being, and a lovely actor.
David Marcus needed to look believable as the son of Carol and Kirk. Merritt Butrick fit the bill.
[O]n a physical level I think what I liked was that his hair was the same color as hers, but it was curly like Bill’s, so I thought, “Well, that’s plausible.”
The biggest casting coup was giving a young Kirstie Alley the role of Saavik.
“She had no experience whatsoever,” Meyer told Enterprise Incidents, but in his interview with Star Trek: The Magazine he remembered Alley telling him she was so enamored of Spock as a child that she wore her Vulcan ears to sleep.
She didn’t have to find the role; she didn’t have to work her way into it. She’d been living it somewhere in her head for years. There just wasn’t a contest. I don’t recall seeing another actor for that part who was as persuasive.
The studio kept trying to make it more of a “tits and ass” performance, as Meyer put it in his interview with Enterprise Incidents.
I said, “No, no, no.” That’s real. You’re in the navy. You’re a pro. Just do your job. You’re good, you’re at the top of your class here.
Alley also brought what Meyer described to Star Trek: The Magazine as an “otherworldly quality” to the role.
She was also able to encompass that sort of flat unemotionality, but she’s basically a comedian. What I didn’t konw was that that flatness, like Leonard’s, frequently comes out of a kind of a deadpan. I realized that when I watched her doing it. Then, at the other end of it, there she was at Spock’s funeral, weeping. I remember somebody came running up to me and said, “Are you going to let her do that?” And I said, “Yeah,” and they said, “But Vulcans don’t cry,” and I said, “Well, that’s what makes this such an interesting Vulcan.”
Sources for this story include: Dennis Fischer, “From Behind the Scenes of The Wrath of Khan,” Enterprise Incidents 16 (April 1984) 43-47 and Star Trek: The Magazine 3, #5 (September 2002)