By smaller villages I take it that you mean, Yudanaka and Nozawa onsen?
You can use your Rail pass to travel to Nagano and from there take a Private Railway (Nagano Electric Railway) to Yudanaka this last part costs (¥1,130 Yen). Here is an example of the journey.
http://tinyurl.com/ayqz98l
As for Nozawa, take the private rail back to Nagano and from there the JR Pass will cover the rest.
For example see http://tinyurl.com/d4ptyps
If you plan your trip to Nozawa using Hyperdia make sure to use the station name TOGARINOZAWA-ONSEN (and not Nozawa) or else you might end up somewhere completely different!
There is no exchange office at Itami (are you sure you are flying to Itami? Most international flights land at Kansai International Airport). The closest ticket exchange office would be Osaka station or Shin-Osaka. Here is a list with exchange points. http://www.japanrailpass.net/eng/en009.html
Let me know if you have more questions!
-Daniel