The city of Sirmione is located on a narrow strip of land extending more than two miles into the southernmost part of Lake Garda. At the end of the Middle Ages, during a period growing communes in Italy, Sirmione was governed by the powerful Della Scala family of Verona whose members were known as Scaligeri.

During the second half of the 14th century, the ruling family ordered the construction of a fortification on the promontory of Sirmione. The castle served a defensive function, but also acted as a small port with its docks serving as a safe haven for Scaliger and Venetian fleets. The entire castle, even the dock, is surrounded by towers and battlements. The structure was enlarged and improved upon during the following century when the area was governed by the Republic of Venice. The castle remained the most important fortification in the region until the 16th century when a new fortress was constructed in nearby Peschiera del Garda.

In the following centuries, the castle was used as an armory and military barracks. After the unification of Italy, the offices of the municipality of Sirmione and the local police station were placed in the castle. It also served as a small jail. The castle was restored after World War I and is now a museum.