A couple of years after I left the editorship of Gramophone, I heard a story that fired my imagination. One of the things I had left the magazine to do (alongside working with musicians, something I deeply enjoy and my main occupation these days) was to carve out some time to write what I really wanted to write. Which sounded good - liberating - except that I didn’t know what that thing was. I wrote the odd article, some I remain very proud of, but it wasn’t until the mandolinist Avi Avital happened to relate to me, in passing, the tale - almost a myth - of Jascha Heifetz and the stone quarry, that I found the story I would work on almost every Sunday (my ritual) for the next few years. And gradually it became the play A Walk With Mr Heifetz, which at the time of writing is just about to have its Off Broadway world premiere, with the renowned New York theatre company, Primary Stages. It was, however, a story that would have embedded within it a perplexing dilemma - in a play about music, how much music do you actually want?

This is a play about a great many other things as well - but all filtered through conversations about music (by the same token, when the characters talk about politics or fate or personal tragedy, they often use musical images). The (true) story, as told by Avi on that long-ago afternoon at Cafe Rouge in Heathrow Airport, was that Heifetz had visited pre-Israel Palestine in 1926 when, the country not yet having any concert hall, he had played at a stone quarry by a kibbutz called Ein Harod, in the North. After the concert - which thousands had crossed the country to see - the brilliant yet eccentric composer and kibbutznik Yehuda Sharett (brother of Israeli founding father Moshe Sharett) somehow persuaded him to go for a walk. The pair walked and talked all night. And the play takes this event, and its resonances for both Sharett brothers and for the creation of the new state, as its subject.

And I was pretty sure, almost adamant in fact, that I knew how to handle the music right from the start. As in, there would hardly be any and what there was would be piped in. That meant, some atmospheric music at the start, to give a sense of the concert that had just been heard, followed perhaps by some Jewish folk music, something similar to close the act, and then again musical bookends for Act Two. There would be one piece of music-making at a pivotal moment by one of the characters, but that would be all. I even had an unopened violin case on stage throughout just to underscore the lack of, well, score.

This wasn’t to be perverse, still less to play with the audience’s expectations (one can almost imagine the self-consciously wry comments of Private Eye’s ‘The Critics’ comic strip). It was because I love Peter Hall’s published diaries, and have always remembered what the revered director wrote about putting together the first production of Peter Shaffer’s Amadeus at the National Theatre, which he then ran. He knew that the single biggest danger to the play was the music of Mozart. Which might sound odd, given that Amadeus is, of course, about Mozart. But in Mozart’s music the art form achieved a perfection. Hall knew that, just as he knew that mere words could never hope to measure up. Just as rival composer Salieri, whom Shaffer made the audience’s eyes and ears (and finally Mozart’s nemesis), talks in the play about his sense of his own art wilting before Mozart’s genius, so Hall instinctively sensed that, too much of the music in that play, and the play itself would melt - Icarus to Mozart’s sun.

Jascha Heifetz was not a composer, but as an instrumentalist he also achieved as close to perfection as any human being can. What do we mean by that? What is perfection for an instrumentalist? I would define it as a sense of striving for an ideal that most can barely envision, and then, at least in some ways and at some moments, achieving that ideal - or at least, giving the sense that that ideal not only exists, but is achievable. That, perhaps, is what a truly great artist is there to do.

And Heifetz did that. And so how could I have characters talking about what had barely been imagined and therefore could barely be put into words? Put the thing itself next to those words, and the words would pale. The drama would become anaemic.

That’s what I thought. And I continued to think it throughout the play’s early private readings. They all, thankfully, elicited very positive audience responses but there would also always be someone who would say, 'Couldn’t we have more music?' And I would always smile inside and think, 'It’s not a concert-play, and it’s not Every Good Boy Deserves Favour' - Tom Stoppard’s tricksy play that was all about an orchestra, who were thus arrayed on stage - and that they didn’t know. But here’s what I have learnt over these last few years. The audience always knows. They might not know how to fix it, but they sense, unerringly, when something is missing.

The great revelation came last year, when I was invited to have a rehearsed reading of the play at the Jewish Book Week Festival at King’s Place in London. In the previous readings, I had actually had a violinist (Bela Horvath) play for 10 minutes before we started, just as a nice way to settle the audience into their seats, but it was always separate from the play itself. I had planned the same for King’s Place, but the play was running long, so there wouldn’t be time. However, I already had a violinist - Simon Hewitt Jones - so had to use him. And so we started to work on just letting the solo violin dip into the play, only here and there. And it worked. The splendour of the violin sound gave flesh to the words.

And everyone involved with the play who heard it told me that there had to be more. And this time I knew that they were right.

But the breakthrough in terms of how to do it came when Primary Stages’s artistic director (and now director of the play itself) Andrew Leynse asked whether our violinist could not in fact be female. And that made sense. Because - I should explain - the play features three male characters and no women. Yet so much of it is about women and their effects on these people, suddenly I realised that our violinist could somehow embody these women, as well as the millions more whose presence is somehow felt but never seen. And there was the difference. No longer were we wrestling with the question of how to compete with Heifetz, or how to embody him on stage. Now the musician had a reason for being there, a point, a character. Not a specific character but an abstract, just as music itself can embody a multitude of reasons and emotions and often all at once.

We were lucky to find a brilliant young violinist in Mariella Haubs, who not only is a protege of Itzhak Perlman (and a frequent collaborator with Joshua Bell) - so musical pedigree assured - but who has a very specific and assured stage presence. And once we found her, the right violinist for our sense of what the music could be on the stage, everything flowed. And now we have a great deal of music in the show, and the play in many ways pivots around it.

Not that Peter Hall was wrong. If music hasn’t got a specific way of working in harmony with the speech, if either the playwright or a director or both cannot find a language where that is not only doable but vital, then yes, it could be a wrecking ball. And the funny thing is, now that I think about what I know of Hall’s Amadeus, he actually did end up including quite a lot of Mozart’s music. While the latest NT production included a full orchestra on stage - and that didn’t come out of nowhere, it was built on the intensive early work that Hall and Shaffer did for that first production. They simply made sure that the music became a character. A character that, somehow, is the play itself.

Neat trick, as the Americans would say. And it only took me three years to understand.

'A Walk With Mr Heifetz' is playing at the Cherry Lane Theatre in New York until March 4, presented by Primary Stages. More information here: cherrylanetheatre.org