Inoue, 29, recently officially vacated all of his bantamweight titles. In May, the Japanese will fight the American for the world champion belts in his fourth weight category.
In his last appearance in the ring, Fulton won on points against Daniel Roman. Inoue last fought in December, knocking out Paul Butler.