Scotland's version of the smoking ban has led to ructions at the Edinburgh Fringe, with actors threatening to defy the ban by lighting up on stage, and shows being changed, writes Brian Logan.

To Mel Smith, it's Nazism. To Tomek Borkowy, who runs Edinburgh's Hill Street Theatre, it smacks of the totalitarianism against which he fought in his native Poland. In Scotland, smoking on stage is forbidden.

What's raised heckles at this year's Fringe is that, unlike other anti-smoking legislation from New York to Ireland, the Scottish ban does not exempt actors on stage or screen, or even allow for the use of herbal cigarettes. After all, says William Burdett-Coutts, who runs the Assembly Rooms, "there is such a thing as air-conditioning. I never heard of anyone feeling uncomfortable or getting sick after seeing someone smoke on stage."

But the Scottish Executive (or the Tartan Taliban, as theatre people are saying) has taken an absolutist approach to clean air. "The smoking legislation aims to protect the public from the harmful effects of second-hand smoke," runs its official line. "This applies equally to actors, performers and theatrical audiences as it does to other workers and members of the public."

In a country where 13,000 people die every year from smoking-related illnesses, this might seem sensible. And local support for the ban has risen from 56 per cent to 78 per cent since the Smoking, Health and Social Care Act came into force in March. But Scotland had reckoned without the Edinburgh festival, where the appetite for controversy is insatiable, and where artistic freedom is as sacred as the right to a pint at 5am.

Leading the protests is the unlikely figure of Mel Smith. Smith appeared as Winston Churchill in Mary Kenny's play Allegiance at the Assembly Rooms, and of course Churchill was partial to cigars. So is Smith, and he wanted to smoke on stage while playing the wartime leader.

Smith told an Edinburgh paper: "Adolf Hitler was anti-smoking. You couldn't smoke at Adolf Hitler's dining table, so he'd be pleased, wouldn't he? Congratulations Scotland."

Smith threatened to defy the ban during the play's run. The authorities promised "to take the appropriate action [ against him], if it is necessary to do so," said councillor Sheila Gilmore. Council officers were present when Smith managed to get through the play's opening performance without lighting up. At the official launch of the Fringe, the day before, he smoked a cigar inside the Assembly Rooms for the benefit of photographers. Onlookers held their breath, partly to avoid the smoke, partly in expectation that Smith would be clapped in irons. But, as Fringe director Paul Gudgin says, "if you're an enforcement officer, you wouldn't do it at a press conference, would you? You'd be sneaking round with a letter later on."

According to Gudgin, there are as many supporters as opponents of the ban among Fringe participants. "There are a lot of people who feel very strongly that it's a freedom of expression issue. But there are other performers who welcome the ban. After all, so much of what you do on stage has to be a representation. Why should smoking be any different?"

This is one of the pro-ban camp's arguments: if you can simulate sex or drug-taking on stage, why not smoking? Councillor Gilmore says: "Actors can evoke all kinds of things without actually doing them. That's what acting's about. We don't demand that they draw blood on stage when they fight."

The official line from the Scottish Executive is that "if smoking requires to be represented in film, TV and theatre performances, realistic alternatives can be used (artificial cigarettes) or developed, if the industry feels they are not suitable." But Scotland's ban forbids even herbal cigarettes, or indeed any "lit substance" in an enclosed public space. The only remaining alternative, say protesters, are unconvincing fake fags, with their billowing plumes of talcum powder.

So why didn't Scottish legislators, like their counterparts in the US and Ireland, either exempt the performing arts entirely or allow the use of herbal cigarettes on stage and on film sets? According to Tomek Borkowy, director of Hill Street Theatre and a high-profile campaigner against the ban, "the city council told me that herbal cigarettes couldn't be allowed because it would make the ban too difficult to police. But that's not my problem, that's an administrative problem." Borkowy insists the legislation was poorly planned, and that the Executive "didn't research the impact it would have on the arts industry. And now they know that they f**ked up."

Gudgin, who has been working with the city council on administering the ban, believes that "the Executive knew it was going to be controversial and that they had a battle on their hands, and so they were determined to make the ban as complete as possible. The more exceptions and loopholes you create, the harder it becomes to enforce."

He is hopeful that when the law comes up for review within the next year, "we can make a good strong case for some kind of exemption". But, at least publicly, the Scottish Executive shows no signs of budging. "It's not something we're going to reconsider," said a spokesperson. "We're sticking to our guns on this."

So how drastic is the effect on the festival? Several productions in Edinburgh had to be redirected to accommodate the ban. The Unprotected, an unflinching piece of theatre about the lives of Liverpool prostitutes, has transferred from the Liverpool Everyman to the Fringe. Its characters smoke; talcum powder wouldn't do. The director, Nina Raine, has called the Scottish ban "outrageous", but has had to bow to the inevitable, ditching the fags and having the characters drink endless cups of tea.

In Red Shift's acclaimed original production of Get Carter, "everybody smoked their heads off all the way through," says director Jonathan Holloway. But not in Edinburgh. Likewise the Fringe hit Bill Hicks: Slight Return, the poster for which depicts actor Chas Early as the late stand-up with a cigarette drooping raffishly from his mouth. The show was returning to Edinburgh, its pre-publicity claimed, "in defiance of the smoking ban". But reality bit, and faced with the threat of a £200 (€300) fine, the posthumous Hicks has had to stub out the ciggies.

Where once there was innocuous on-stage smoking, what has now emerged is a cat-and-mouse ritual whereby performers tease audiences with the prospect of an illicit puff. In Talk Radio at the Underbelly, actors are toying with cigarettes, threatening to light them, turning their backs to the audience and - are they having a sneaky toke? Fags have become as charged on stage as loaded guns. This will-they-won't-they? ceremony has become the most bizarrely compelling sideshow on this year's Fringe.

All a little silly, you might think - but there are artistic issues at stake. According to Burdett-Coutts, whose venue hosts Mel Smith's play, "the fact that someone playing Winston Churchill cannot smoke a cigar on stage is absurd". It's symptomatic, says Borkowy at Hill Street Theatre, of British Philistinism towards the arts. "You may be able to simulate sex. But if you start to simulate smoking, and instead of inhaling you have to exhale, it's completely idiotic. I come from a country where theatre is taken seriously. I have been trained in the Stanislavsky method. You cannot cheat in this way. Imagine trying to show smoking on film, with powder! Who is going to watch this kind of film?"

Certainly, the ban poses greater problems to film. Gudgin cites Trainspotting, with its scenes shot in smoke-filled pubs, as an example of a movie that would now have to be shot elsewhere. But the national film body Scottish Screen is playing things cool. Their spokesperson tells me, "there's always special effects".

Another artform with a lot to lose is stand-up comedy, the iconic image of which - pace Bill Hicks - is a performer with a mic in one hand and a fag in the other. I spoke to the comic Reginald D Hunter, who is rarely seen on stage without a cigarette. But he welcomes the ban, which he thinks will help him give up smoking. Doug Stanhope is characteristically less sanguine. Smoking on stage, he claims, directly affects the quality of his comedy. "It fires me up. It keeps my attention deficit to a minimum. Without a cigarette, my train of thought isn't as tight. All those long awkward pauses in my act, that's me mentally reaching for a cigarette. And without a cigarette, I'm less able to depict the life of excess that is the subject of my comedy."

As far as he's concerned, saving Scottish lives is no compensation. "If you want to save people's lives, give them something to live for. Make the place less depressing."

So will anyone make themselves a martyr for the performer's right to smoke? "The trouble is," says Stanhope, "you don't make yourself a martyr, you make the venue a martyr." Responsibility for enforcing the ban falls on the licensee, which is why Assembly Rooms chief Burdett-Coutts is so anxious about Mel Smith's refusenik threats. "The worst-case scenario is a fine of £5,000. But what can I do? And at the end of the day, he's Mel Smith. He'll do what he likes." In the end, he didn't smoke on stage, but his character took out a cigar and made a reference to his doctors telling him to stop smoking, followed by a meaningful look at the audience

In England, public health minister Caroline Flint is consulting the performing arts industry before drawing up anti-smoking legislation, and has suggested that an exemption is likely. In Scotland, Gudgin remains confident that, once the Fringe is over, a similar compromise will be reached. "The theatre lobby in Scotland will continue to try to get one or two accommodations," he says. "The Fringe hammering away on its own is not going to effect major change. But the whole film and theatre industry together may achieve something."

• Guardian service