Japanese people often understand English, but only if you say the right words. There are a couple of reasons behind this.
The first reason is that everyone is taught the same English in public schools. Secondly, the Japanese "borrowed" words from English without consulting any of us, so the result is like an alternate-dimension version of English.
Lucky for me, I used to be an English teacher in Japan, so I got pretty good at this alternate language. There's a list of words below if you want to dive right in to the vocab, but you might consider reading these communication tips first:
Use as few words as possible. Put important words up front.
Don't ask, "Can you tell me where the hotel is?" because it's too many words for your average Japanese person to process. Instead, make a polite face and ask, "Where is the hotel?"
Use gestures to augment your speaking.
What someone sees can be just as valuable as what they hear. Nod your head in agreement or disagreement. Point in directions. Make an X with your hands to either say "Not doable" (or "We're finished," if you're in a restaurant).
Give speakers the impression that you're following very closely.
If you're listening to a longer bit of speech, like directions to someplace, acknowledge every clause with "uh huh" or "OK." If you stay quiet and let them speak, they'll start thinking you're not paying attention.
Repeat things often.
Japanese speakers do this all the time. Upon being told a place is closed: "Oh, it's closed?" Upon being told that the neighbor's daughter is getting married: "Oh, really?"* If you receive directions from someone, it's normal to run through them to confirm them.
*Fun fact: This is where the "ah, so" stereotype comes from. Emperor Hirohito was trotted out of the imperial palace to meet the people after World War II. But after all that time in the palace, he had no social skills. People were eager to meet him, but all he could manage to say to anyone was "Ah, so?" (Oh, really?)
The vocabulary list
Keyword English is easy. It's just a short list of word substitutions, and the whole thing is printed below. When you're done, you can move on to changing hotel reservations on the fly.
Instead of: bathroom
Say this: toilet
Instead of: hang on
Say this: please wait
Notes: Gesture this instead by putting your hands near your chest, palms facing outward (as if making a "stop" or "whoa" motion)
Instead of: I'm ____.
Say this: My name is ____.
Instead of: I have a [headache / backache / etc.]
Say this: My [head / back / etc.] hurts.
Instead of: Huh? / What? [in case of a difficult accent]
Say this: I don't understand.
Instead of: arcade (video games)
Say this: game center
Instead of: cyber cafe
Say this: Internet cafe, or just "comic"
Notes: People often read manga (anime-style comics) at Internet cafes, hence the "comic."
Instead of: carry-oh-kee
Say this: kah-rah-oh-keh
Notes: This is the correct pronunciation of "karaoke." I still say "carry-oh-kee" around my Western friends, but I say "kah-rah-oh-keh" around Japanese speakers.
Instead of: sah-kee
Say this: sah-keh
Notes: Like "karaoke," this is a pronunciation thing for "sake." "Sah-kee" sounds like "saki," which means a variety of other words unrelated to booze.
Instead of: notebook
Say this: note
Instead of: laundromat
Say this: coin laundry
Notes: The Japanese pronunciation of 'laundry' sounds more like 'laun-do-ree,' so be careful.
Instead of: laundry detergent
Say this: soap
Notes: "soap" still applies to ordinary soap, as well.
Instead of: conditioner (hair)
Say this: rinse
Instead of: one's own [car, bag, house, any noun really]
Say this: my [car, bag, etc.]
Notes: This one's kind of backwards. The Japanese use the word "my" without associating it with "me" - so as a result you might have a "my car," or a "my bag." Just be aware of that, because it really confused me a few times.
Instead of: Information Center / help counter
Say this: information [just that word]
Notes: For the Japanese, the English word "information" refers to a place with guides, not knowledge or facts.
That lesson wasn't too bad, was it? Now you're ready to communicate with Japanese people! Isn't that cool? Why not put your new skills to use and book a better hotel room?