Milkstars Sean Penn in the performace of his career as America's first openly gay politician Harvey. Michael Dwyermeets the movie's Oscar-nominated makers: director Gus Van Sant and screenwriter Dustin Lance Black.

HARVEY MILK relished his distinction as the first openly gay man elected to civic office in the United States. He was 47 when voted onto the board of supervisors in San Francisco. A year later, on November 27th 1978, a colleague shot and killed both Milk and the mayor, George Moscone.

Pleading “diminished capacity”, the assassin, Dan White, had his charge reduced to voluntary manslaughter and was sentenced to seven years in prison. The lenient verdict triggered riots on the streets of San Francisco. In 1985, after White was paroled after serving less than five years, he killed himself in his garage.

Earlier that same year, Rob Epstein's enthralling The Times of Harvey Milkwon the Oscar for Best Documentary, piquing film industry interest in a dramatic treatment of Milk's eventful life and tragic death. Oliver Stone and Rob Cohen were attached to projects at different stages. Openly gay director Bryan Singer planned a film based on Randy Shilts's book, The Mayor of Castro Street. The central role was offered to Tom Cruise and, perish the thought, Robin Williams.

Now, finally, Sean Penn plays Harvey Milk, with James Franco and Diego Luna as his lovers and Josh Brolin as Dan White in Gus Van Sant's incisive Milk, based on a meticulously researched screenplay by Dustin Lance Black who, like Van Sant, is gay.

When I met Van Sant and Black in London last weekend, the director said he had wanted to make a movie about Milk since the early 1990s.

“I wasn’t aware of Harvey until he was killed,” he says. “I was living in LA at the time and I wasn’t out then. I became really interested in Harvey when I saw the documentary. I had just made my first movie, Mala Noche, which had a gay theme about a man chasing after a young Mexican migrant. All of a sudden I was going to film festivals and I got to know other gay film-makers – Derek Jarman, Rob Epstein.

“I also got to know the Castro district of San Francisco very well because it has a gay and lesbian film festival and I was there quite a few times with my films. Then I heard that Oliver Stone was planning a film about Harvey Milk but had decided against it. I became the replacement director on that project for about a year, but it didn’t pan out.

“A few years later, I wrote a script based on Harvey. In 1998 I offered Sean Penn a role in it. I thought it could have been done in an allegorical way. My idea was for a film about a fictional guy who has a camera shop in Portland, Oregon and runs for the city council. I thought it might be more interesting to reinterpret the story that way. Then Lance eventually showed up in 2007 with his screenplay and I wanted to direct that.”

Dustin Lance Black was three years old when Harvey Milk died. He found it tough growing up in a Mormon family in conservative Texas, complete with a stepfather who was in the military.

“It was such a homophobic atmosphere,” he says. “The only words I heard about gay people were derogatory, so you felt that you’re less than the other kids and that you’re going to hell. I retreated very much into myself. I became very shy and stopped socialising.

“It would have been impossible for me to see the documentary on Harvey back then. It certainly wasn’t something my mom would have rented on VHS. I felt very fortunate when my stepfather was transferred from San Antonio to an army station in the Bay Area. That was a totally different world — and such a wake-up call for me. I fell in love with the theatre scene, and there were gay people in it. That’s where I first heard the story of Harvey Milk, which was a revelation for me. It was such a story of hope that it blew me away.”

Black finally got to see Epstein’s documentary. He also met Cleve Jones, a close friend of Milk and fellow gay activist, played in Van Sant’s movie by Emile Hirsch.

“Cleve took out boxes full of all these old papers of Harvey’s,” says Black. “And he showed me a different side of Harvey, that he was a horrible boyfriend to have and that he never made a dime running his camera shop because he was too busy flirting with guys out on the street. Until then, I had envisioned Harvey as a saint. All this new information made me realise he was human just like anybody else. That made him even more interesting to me.”

In an uncharacteristically relaxed and charismatic performance that well merits an Oscar next month, Sean Penn inhabits the role of Milk so subtly and completely that he disappears inside the character.

"Sean was interested ever since I first discussed it with him in 1998," Van Sant says. "I didn't really follow up on it then. Then, when I called him in 2007 and told him we were making Milkand I asked if he would play Harvey. He read the script and agreed to do it.

"In the film business, you have to make an offer if you want an actor to commit to a role, and you need money for that because the offer is irrevocable on a pay-or-play basis. Because Sean was interested, the next step was not to let him take another film while we were trying to raise the money. We were fortunate because he had finished Into the Wildand that was going to consume him for another few months."

In marked contrast to the coy treatment of the gay lovers played by Tom Hanks and Antonio Banderas in Philadelphia, Penn and James Franco share several passionate scenes in Milk. Black says that the population of Castro Street turned out to watch their long open-air kiss and that Franco told him afterwards how Penn texted his ex-wife Madonna to say, "I just broke my cherry to another guy. I thought of you. I don't know why." She reportedly replied: "Congratulations."

The movie features newsreel clips of Anita Bryant, the anti-gay former pop singer whose politics were the polar opposite of Milk’s campaigns in the late 1970s.

“I’d love to have met her when I was researching the screenplay, but she’s not really accessible,” Black says. “In my early drafts, the more honest I was in dealing with her, the more of a caricature she seemed. At a certain point I decided it would be better to go with archive footage of her in her own words. It was easier to find humanity in Dan White than in her.”

Black’s screenplay refuses to demonise Milk’s killer, depicting White as troubled, conflicted and frustrated. “It was interesting that Harvey liked him and was disappointed he couldn’t make the friendship work,” says Black. “Even when there was a fissure between them, Harvey continued to reach out to him. As the movie is from Harvey’s perspective, I had to figure out what Harvey saw in him.

“He’s a tragic figure, Dan White. I think he was just trying to prove himself. He came from this family of hero firefighters and every time he tried to be a hero, in the army and the police force, he failed. This was his last big shot. Then there was this guy, Harvey, who stood for everything his family said was wrong, and Harvey was succeeding.”

A key sequence in the film charts Milk's successful 1978 campaign to defeat Proposition 6, a Californian referendum to have gay schoolteachers fired. On election day in November 2008, shortly before Milkopened in the US, a majority of Californian voters approved a Proposition 8, referendum to outlaw same-sex marriage.

“That’s an eerie coincidence,” says Van Sant. “Obviously, we didn’t envisage that when we preparing the movie last January. Nor did we envision the emergence of Obama. That was interesting because Obama was running on a platform of hope, as Harvey was. His speech in Iowa was so reminiscent of a Harvey speech.”

Barack Obama “dodged” Proposition 8 and “dodged it poorly,” states Black, who worked as a volunteer with the Obama team. “I got very involved with his campaign, so it was very upsetting when people backing Proposition 8 were using Obama’s statement that marriage is between a man and a woman. They had taken it slightly out of context, but it was still a stand that he took. That was severely disappointing to the gay community because we had backed him so strongly.

“Then he chose Rick Warren to give the invocation at his inauguration. Warren has been very vocal with his views about gay people, comparing them with paedophilia and incest. His website explicitly states they should be excluded from Christianity. That’s not very Christian and, frankly, it seems totally out of line with Obama’s message.

“It’s a lesson for the gay and lesbian community that has been lost to an extent since Harvey Milk’s time, and that is to demand full equal rights and not to settle for the crumbs and the small steps. It’s important that we don’t let that happen under the Obama administration.”

• Milkopens today

Reel lives: making sense of history

Gus Van Sant's quietly powerful film Elephant, which earned him the Palme d'Or at Cannes in 2003, was inspired by the Columbinehigh school shootings that claimed 13 lives. His intriguingly elliptical Last Days(2005) featured Michael Pitt as a reclusive rock star based on Kurt Cobain.

"When I dealt with real-life stories before, my approach was more oblique," Van Sant says. "I never used real names in Elephantor Last Days. I knew that Elephant was going to be looked on as a film about Columbine, so it didn't matter if we named the high school or the kids.

"Yet it's not really the same thing, which makes it easier to deal with the subject. You know, they're still withholding all the information they got on the Columbine kids and what they wrote and recorded. So we were dealing with an unknown period of time and with missing pieces, just as we were on Last Daysbecause nobody knows what happened over those last three days of Kurt Cobain's life.

"It was very different with Milkbecause we were dealing with well-documented momentsof his life and using his real name and the real names of other people. I always look on it as like a pantomime of the real thing. Your interest in it is watching the playing out of a historical moment rather than something that is real."