The earliest mention of Vrbanj under the name "Varbonj" dates back to 1331.
At the location of Hum above the village, there is a stone mound, likely a prehistoric structure, next to which are the remains of the pre-Romanesque church of St. Vitus.
In the area of the village itself, Roman remains have been found. The plateau among the hills in the central part of Hvar is clearly an interesting natural constellation for human habitation.
With 1400 inhabitants in 1900, the not-so-large plain among the hills created the largest inland settlement on the island of Hvar. Today, Vrbanj is home to about 400 residents.
From the edge of the village on the northern side, there is a view of Stari Grad Plain, while on the southern side begins the gradual ascent to the highest peak of Hvar, St. Nicholas, at 626 meters.
In the 15th century, the inhabitants of Vrbanj founded the village of Vrboska on the coast, 4 km away, and began engaging in shipbuilding. In 1475, also in the 15th century, Vrbanj got its own parish priest, the first local parish priest on the island of Hvar.