Japan, host of the '64 Olympics was really starting to make strides as the world's second most vibrant economy. A monorail train connected Haneda with greater Tokyo. Upon clearing Immigration and Customs and upon meeting my sempai who had come to greet me - well the first order of business was to do mine!
I walked thru an open door and was hit by an overwhelming stench. Dark, dank and stingy, these public toilets were a huge business opportunity for Lysol Co in the land of "WA" (和) or harmony.  The assault on my olfactory senses aside, I became aware as I was standing there that I was being stared at by an elderly female cleaning attendant. With jet lag I did a quick inventory. Japan wouldn't have men's urinals in the women's room so obviously I was in the right place. Later I would come to understand that this behavior would happen wherever I traveled in Japan.
To the Japanese at large, foreigners were still not a common sight away from military bases. On trains young children clinging to their mothers' skirts would peek or stare and point at us sometimes in bewilderment, and sometimes in fear about the "gaijin" [外人]. This was a not so kind term from a Westerner's point of view for an alien more akin to a Martian than a person from another culture. Back then we were issued "Gaijin" identity cards that we had to keep with us - ever the outsiders.
Very few signs were in English let alone the myriad languages that adorn walls on mass transit routes today. I treasured my English version of my subway map of Tokyo a heck of a lot more than the few yen in my wallet!
Over time we do start to assimilate however. We learned how to press into crowded subway cars, position ourselves closest to the exit and generally how to navigate through crowded stations. True mastery came with training or perhaps after training. Bone tired, being able to sleep on the train and wake up just before your intended stop and avoiding the hassle of eye contact was a wonderful mental break from the here (hear) and now. Late at night the last place you wanted to find yourself was out in the burbs or even the countryside well rested and stranded after the trains had stopped.
Unlike other countries that I've visited I also learned to avoid eye contact with fellow sojourners. Usually easy to identify, no friendly effort was made by the old timers towards those newly set out on a mysterious adventure in Japan. We kept to ourselves. 
Today, after countless return visits to Japan thru the newer gateway of Narita airport, we're coming full circle as Haneda again accepts international flights. Today, Japan's population is used to Westerners in public and we can travel without a stir. Signs are clear and in English as well as numerous other languages. And the bathrooms would do Star Trek sci fi writers proud. I wouldn't go so far as to say that Westerners now acknowledge each other on the streets. 
1 "WA" or great harmony was a confused interpretation of the 倭人"wajin" as addressed by the Mongol courts to Japan. The Mongol envoy lost his head when the characters selected actually meant "hairy barbarians". I guess some things are better heard rather than seen. This precipitated the 2 Mongol invasions.
2 My guess is that because Japan is an insider/outsider society that as foreigners we would never quite fit in but still tried and with seniority we felt more inside than those that were newer to the mix. The Japanese culture is also about collective responsibility. No one wanted to get into any kind of difficulty that would be problematic for any group that one might associate with.
3 One of my favorite movies about Japan is called "Mr. Baseball" with Tom Selleck. There is a scene where someone has to show him how to use the more traditional Japanese toilet.