A serious plot flaw in the Dark Knight

I've previously noted several things I hated about the Dark Knight. Here's one more. (Spoiler)

The Joker arrives at the fundraising party at Bruce Wayne's penthouse for Harvey Dent, scuffles with Batman, and then manages to get Batman out of the way by throwing Rachel off the roof. OK, I accept that the Joker has incredible intel and knew where Dent was. I even accept that Rachel was miraculously not smashed to pulp by the fall because she was somehow magically cushioned by Batman.

But why, after getting Batman out of the way, does the Joker politely leave? And why does Batman show no urgency in getting back to the party to make sure everyone's ok? It's as if an entire scene was cut from the movie.

*Find vintage Batman toys and collectibles at eBay.