Silba was settled as early as Roman times. In Pocukmarak Cove, a coin of the Roman emperor Antonio Pio from 161 A.D. was found. According to some references the island was settled by the Croats in the 8th cent. Permanent settlements started developing on the island at the end of the 15th and beginning of the 16th cent.

The 9th century emperor and travel writer Constantin Porfirogenet mentioned Silba for the first time in history using the name Selbo (from silva, Latin word for forest). Silba has been a part of to the municipality of Zadar since the year 827. At the request of his sister, the abbess Čika, the Croatian King Petar Krešimir gave Silba to the monastery of Saint Mary in Zadar in 1073. Later the island fell under Venetian rule and in 1639 the Venetians sold it to Captain Fani Soppe.

Silba was then taken over by the famous Venetian family Morosini, retaining the possession of Silba until the first quarter of the 19th century. In 1838 the Morosini sold Silba to the rich American returnee, Mark Ragusin from Veli Lošinj. Due to a 13-year-old tax dispute with the locals, Ragusin decided to sell the island to the inhabitants themselves, and each inhabitant was to pay proportionally for his possession, all amounting to 5025 Bavarian thalers. News of the purchase arrived on the island on March 19, 1852, and in remembrance of that event the inhabitants of Silba celebrate the feast of Saint Joseph, which takes place on that day, every year.

The period of prosperity for Silba was in the 17th and 18th cent. It was the locality of seamen and owners of hundreds of sailing ships. Strong seafaring business conditioned the development of an above-average culture in the way of living, dressing and home decorating. There were more than 1200 inhabitants on the island at the time. The island of Silba today is under the administration of the city of Zadar.