Situated at the main junction of Rhossili and Port Eynon on the B4247 is the village of Scurlage. It was during the 14th century that Sir Herbert Scurlage, who was related by marriage to the Mansel family, built a fortified Manor House known as Scurlage Castle or ‘Scorlayges Kaestel’ approximately a quarter a mile away from the village. It was common practice in these times for those families wealthy enough to own quantities of land to need to fortify their grounds and protect them with force – the Gower castles of Oxwich, Landimore and Weobley are similar constructions. However, Scurlage Castle has not survived the ravages of time and, today, is nothing more than the site of a farm.

During World War II, the village became host to an American Army camp – the Forces being attracted to the region by its proximity to the beaches of Oxwich, Port Eynon and Rhossili. These were all seen as being perfect for troops to stage practice landings as they closely resembled the beaches of Normandy. To this day, the lane from Scurlage down to Oxwich (off the A4118) is still referred to by many locals as American Road. After the invasion of Normandy in 1944, the camp was used to secure Italian prisoners of war.

Modern day Scurlage is home to a large housing estate, a sports club, a strawberry farm, a public house and a medical centre.