In 1853, the Tennessee, a clipper ship ran across rocks and sank in the fog near the Golden Gate Strait. Now it reportedly passes under the Golden Gate Bridge, without a crew, sailing through the waters on dark foggy nights. In 1942, the USS Kennison passed by the ghost ship, and sailors and crew reported seeing the ship and watching its wake, yet nothing was registered by the Kennison’s radar. The Tennessee seems to always appear on fog shrouded nights, its decks are always deserted, and it fades away about a minute after first appearing.
This is a real-world example of a ghost story, loosely connected to a real-world landmark which has plenty of urban legends and myths purported about it. Now imagine what the context of such a bridge would be in a fantasy setting.
First off, an enormous bridge would require wizards to assist in construction, and learned sages would be on hand. Even if the total construction of such a bridge only took 4 years, and cost 35 million gold pieces, the death toll might look different, perhaps be quite high, unless plenty of high level clerics were on hand to assist with injuries. Suicides might be a problem, but in the medieval world resources would not be diverted to stop people from committing suicide, and in a magical setting where painful death can sometimes lead to undead, the shoals underneath such a bridge might become known for their wailing dead, or carnivorous scavengers.
In a low magic setting, certainly more deaths would occur around the bridge and perhaps construction would be delayed several times, pushing up costs. A very wealthy kingdom would see to it that construction is completed, even if the bridge ends up being worthless. And in a high magic setting, such a bridge could probably be constructed in a fortnight with little to no calamitous occurrences during construction. If a war breaks out, perhaps one side decides to destroy the bridge. What kind of spirits or deities might they upset by doing so, or perhaps the bridge is not so vulnerable to attack by being protected by multiple magical wards and armors.
Always, there will be suicides followed by rumor and legends of ghosts, if not actual ghosts.