Watching this movie without being familiar with the characters and story of Shakespeare's play, "Hamlet" is like watching a ballet without having a clue as to the story. It is not impossible to enjoy without knowing "Hamlet", but it makes the experience ever so much better. Note that some productions of "Hamlet" give the Rosenkrantz and Guildenstern subplot little attention, and may even cut it out entirely. See Kenneth Branagh's "Hamlet" to get the full picture (it's a darn good rendition of the play as well.)
The biggest single plus for the film is that it does not only tell the backstory of two minor characters in "Hamlet", played marvelously played by Gary Oldman as Rosenkrantz and Tim Roth as Guildenstern. It also tells a lot about the backstory of the troupe of traveling "tragedians", the company commissioned to put on the play which Hamlet hopes will wake the conscience of the king. This supplies the third lead character, "The Lead Player" played by Richard Dreyfuss.
When either Dreyfuss or other Hamlet scenes are on the screen, everything seems about right. There are even some of Hamlet's more famous scenes played out, with Shakespeare's words. But when the two title characters are along, a sense of deep unreality sets in. For example, at the very beginning of the film, there is a conceit where Rosenkrantz finds a coin in the road and starts flipping it over and over, and it continually comes up heads. Guildenstern bets against that several times, and loses every time, until they reach a count of over 100 heads. Something about their world has become unstuck. It is to this play what the appearance of the specter is in "Hamlet". Something is not right in their world. It's the kind of unreality we sense in Peter Greenaway's films, especially because the visual plays such an important part. Contrary to Roger Ebert, there are visual clues which would be missed on the stage.
Over half of the movie rests on the shoulders of the name leads, and for an audience who is used to seeing Oldman as the perfect villain or Tim Roth as the small time robber "Pumpkin" in "Pulp Fiction", these roles really show a new side of their talent. Roth has the same English accent as "Pumpkin", but Oldman is in an entirely new place. I could hardly take my eyes off him. They are innocents in a chaotic world which is destined to crumble around them.
Since so much depends on the dialogue between the two, some professional reviewers have said the play was unsuitable for the screen. I would counter that what the camera gives us is close-ups of the faces, especially of Oldman and Roth, and captures the most subtle expressions, which would have been missed from the mezzanine of a large theatre.
Let's be fair. This movie asks a lot of its viewers, in addition to the need to see Hamlet. It is not for everyone. But, for those who really like subtlety and a unique kind of mystery, this is for you.