I think the important words are "where possible".
The Picard documentation for this Metada option says,
Translate artist names to this locale where possible: When checked, Picard will see whether an artist has an alias for the selected locale. If it does, Picard will use that alias instead of the artist name when tagging. When "English" is the selected locale, the artist sort name (which is, by Style Guideline, stored in Latin script) is used as a fallback if there is no English alias.
So, translating artist names is only "possible" where there is an alias for the Japan locale. I haven't looked, but I expect there are very few such cases.
It would be nice if MusicBrainz and Picard was able to translate any artist name to any locale, but sadly it doesn't have that ability.