The man's name was not immediately released and there was no immediate word on the charges he would face.
Officers were called to the store shortly after 1:45 a.m. and exchanged gunfire with the man before realizing there were hostages in the store, police said.
The officers left the store and called for assistance. Officers from nearby Gastonia, Gaston County and Mount Holly surrounded the store while negotiations were held with the man.
The hostages were released shortly after 4 a.m. There were no injuries, Wilson said.
Belmont is about 15 miles west of Charlotte.