Police killed three attackers and detained one, ministry spokesman Almas Sadubayev said. Police have surrounded other attackers but some others remain at large.
Sadubayev said police suspected attackers were "followers of radical, nontraditional religious movements", a phrase often used in Kazakhstan to describe Islamist militants.
The three killed were a clerk at a firearms store, a military officer and a serviceman at a national guard facility, Sadubayev said. Nine servicemen were also wounded.
Police have temporarily shut down malls and other entertainment venues in the city as well as public transportation.
Aktobe, an industrial city 100 km (60 miles) from the Russian border, was the site of Kazakhstan's first suicide bombing in 2011 when a local man used an explosive device inside the building of the state security service.
Authorities in the mostly Muslim nation often announce detentions and trials of Islamist militants, but most of them are people who had traveled or planned to travel to places such as Syria and Iraq. Violent clashes within the country itself are rare.