"The children are in our custody and our number one priority right now is their health and safety", New Mexico Children, Youth and Families Department Secretary Monique Jacobson said in a statement, according to AP.
Deputies said three women believed to be the children's mothers were also detained and later released.
Aside from a few potatoes and a box of rice, there was no food or water in the filthy compound, which authorities described as a travel trailer buried in the dirt, ringed with tires and earthen berms and covered in plastic.
The children, ages 1 to 15, looked like "third-world country refugees" and had only "dirty rags for clothing", Taos County Sheriff Jerry Hogrefe said.
Hogrefe said officials were initially searching for Lucas Morten and others, including 39-year-old Siraj Wahhaj, who was wanted for child abduction and 3-year-old AG Wahhaj, a child that was abducted.
The rescue followed a police operation which began late previous year to find an abducted three-year-old child - who was not among the children found at the property.
Clayton County police said in a missing persons bulletin that Wahhaj and his son were last seen December 13 in Alabama, traveling with five other children and two adults. An investigation into his disappearance by the FBI and authorities in Clayton County, Georgia, is continuing.
According to authorities, the two men, Siraj Ibn Wahhaj and Lucan Morton, were heavily armed with rifles, pistols and a large cache of ammunition, and were believed to be Muslim extremists.
Hogrefe says they were the saddest living conditions he's ever seen.
In addition to the child abuse charges, Morten has been charged with harboring a fugitive.
Trump proposes auto gas mileage rollback
It could also become a polarizing issue in elections in November. "We will use every legal tool at its disposal to fight back". Now they're only about one-third, with less-efficient trucks and SUVS making up the rest.
"It's obvious to me that they were brainwashed and feel great intimidation from the men that were in control of this facility", Hogrefe said of the women and children.
Authorities in New Mexico say they served the search warrant after receiving a message from a Georgia detective that they believed came from the compound saying "we are starving and need food and water".
Both Sarij Wahhaj and Morten originally refused to follow verbal commands from the tactical squad but were both taken into custody.
Siraj Wahhaj and Jany Leveille are also said to be married.
On Dec. 29, Clayton County police said they had obtained information that the group could be in the area of Taos, NM.
The last reports of the father and son being seen in public together came on December 13, when they were involved in a single-vehicle accident on I-65 in Chilton County, Ala.
Mr Wahhaj is also wanted on suspicion of abducting his three-year-old son, but the boy was not among the group found at the compound, U.S. media report. Police say the starving children were living with five adults in an underground trailer near Amalia, NM, the BBC reports.
Wahhaj was jailed on suspicion of child abduction.
None of the adults would give a statement as to the boy's whereabouts, Hogrefe said after the raid, but "it was reasonably believed he was there a few weeks ago".
Alabama police said they were told the group were going camping in New Mexico, the station said.