On the rocky coast of Panormos, a village near Rethymnon in nortern Crete, there are the scattered ruins of a castle that used to be one of the major fortifications in Crete.

Panormos (located 20km west of Rethymnon) at the beginning of the 20th century was called Castelli of Mylopotamos or Roumeli Castelli. It is believed that it is built on the site of the Roman town Panormos. Before the 20th century, Roumeli Castelli was one of the most important commercial harbors of Crete, serving the entire Mylopotamos province.

Next to the pier of the village harbour and along the steep coastline, the Genoese pirate Enrico Pescatore built in the early 13th century the fortress of "Castel Milopotamo". When the Venetians conquered Crete, the fortress was restored and a settlement (bourg) started developing around the fort, later evolving into the current village of Panormos.

The egg-shaped fort was oriented from north to south and was equipped with seven towers and two gates, one on the sea side and one on the south side. In 1341, the family of Kapsokalives besieged the fort, in which Alex Kallergis was hiding, but soon quited. Kallergis was a famous Cretan traitor, who joined the Venetians.

In 1538 the pirate Hairendin Barbarossa destroyed Castel Milopotamo, but the Venetians restored it immediately, as it was located in a strategic position. Later the Turks, after having occupied the castle of Rethymno, began walking towards Candia. Thus, when passing from Panormos, they besieged and occupied the fort (in 1647). The Venetians tried to regain it by under the orders of the General Gildasi (Gil d 'Has), but they failed.

Today there is almost nothing left from the fort except a small part of the wall, resembling a pile of stones.