From the first half of the 13th century Swobnica and the surrounding areas belonged to the Order of the Knights Templar, the headquarters of which was in the village Rurka. After the liquidation of the order in 1312 the property of the order was taken by the Order of Saint John, who due to a conflict with the residents of Rurka and the surrounding knighthood were forced to leave the property in Rurka and move to nearby Swobnica.
Learning from their experience in Rurka, the monks decided to build a more defensive structure. In 1377 the Order of Saint John obtained their permit to build. The castle in Swobnica was built at the end of the 14th century. The brick structure was created on the plan of a square and surrounded by a 15-meter wall. Additionally next to the gate they built a 35-meter tower, which strengthened the fortifications of the property.
After the Thirty Years' War the castle was taken over by the Swedes, and in 1680 it was purchased by the wife of the Elector of Brandenburg. Upon her request the structure was rebuilt into a baroque hunting residence. During the modernization works the western segment of the defensive wall was demolished together with the gate, while in the other parts window openings were made. Despite the renovations, the new owners maintained the brick elevation and the tower to which stairs were built from the courtyard (before it was only possible to enter the tower from the shooting porch which was situated at a height of 16 meters). The castle remained in the Hohenzollerns’ hands until 1945.
After the 2nd World War the castle was robbed and then destroyed many times by “treasure hunters” who – among others – broke the chimneys in the castle. During the times of PRL [Polish People’s Republic] the castle functioned as the headquarters of PGR [State Agricultural Farm], which had their offices there and then they opened a granary in the castle. In 1933 the structure was purchased by a private investor from Belgium who was supposed to renovate it. In 2008 the oldest wing of the building collapsed. In 2011 the Banie municipality regained the building and a year later the eastern wing of the castle and the roof were renovated. In 2013 the renovation works of the tower were completed.
The castle is surrounded by 2.5 ha of the natural park from the 19th century. In the park there are numerous species of trees. The castle was included on the list of monuments in 1957. Since 2009 Stowarzyszenie Zamek Swobnica [Swobnica Castle Association] has been operating and its main goal is to rescue the monument.