Is a Japan Rail Pass worth it?

I think you may be looking at Hyperdia wrong. I've just done a search for the same Tokyo - Kyoto trip, and on the bullet train, it would cost ¥13,220 one way - http://goo.gl/5z9LH

Are you sure you selected the shinkansen bullet? I think the cost you found is for the (really slow) normal express trains. For that journey, you certainly want to be using the bullet trains. Tokyo > Kyoto takes 165mins by the Hikari bullet that Japan Rail Pass holders can use, but 488minutes (8 hours!) using the Limited Express trains that I think you may have selected.

In your case, if you only want to go to Kyoto/Osaka, and are there for 2 weeks, it'll be hard to get full value out of a 14 day pass. You may be better off spending some days in Tokyo at either end of your trip, and breaking in the middle for a 7 day train journey down to Kyoto and Osaka. That way the Rail Pass will be cost effective for you, especially if you include your transfer from Kyoto to Osaka and maybe a jaunt out to Nara.

The other thing that's good about a Rail Pass is that you can just jump on trains. It may seem minor when you're in your home country, but once you get to Japan, I'm sure you'll be happy to glide through ticket barriers!

Excellent link queenlain! I've reposted it on our twitter and facebook :)