Police closed off Church Street from Spring Street to South Highland Ave as firefighters responded to the scene.
According to the building department, the fire department was called in after debris was seen falling off the steeple. The building department said once the steeple is secured, a structural engineer will be called in to ensure everything is safe and is no longer a hazard.
Carina Salvador saw the steeple begin to shake.
"It was dangerous," Salvador said. "If it wasn't for the firefighters, it would've tipped over. It's not safe, it's an unfortunate situation. I hope they take care of it."
Click here to follow Daily Voice Ossining and receive free news updates.