Founded in the Middle Ages, the oldest building is the church of St Peter, which was built before the 14th century between the boundaries of Pitve and Vrbanj. In the immediate vicinity, on the Stari Grad Plain, is the ancient Villa Rustica. Matija Ivanić, the populist leader of the Hvar Rebellion of 1510-1514, had a house in Vrboska. The Venetian troops attacked the town in 1510, causing extensive damage. In 1571, the town was again attacked, this time by the Turks, under Uluz Ali. Soon afterwards, in 1575, the church of St Mary was fortified, to become a church-fortress, a unique sight in the islands. The place developed as a fishing community, trading and growing in prosperity. At the beginning of the 20th century, Vrboska had a sardine factory, wooden boat-building industry, a hotel, fishing trade association, and regular boat connections with Split and the neighbouring island of Brač. There was a community hall, library and health clinic.