By Ana Cabrera, Greg Botelho, Chandrika Narayan, Tiffany Ap

(CNN) -- A Utah judge who sparked a public backlash for removing a foster child from the home of a lesbian couple has removed himself from the case.

Juvenile Court Judge Scott Johansen "hereby disqualifies himself from this case and refers all pending matters to be assigned by the presiding judge," a court order issued Monday said.

Johansen initially decided to take a baby away from her same-sex foster parents and place her in a home with heterosexual parents saying the child would be better off.

Facing mounting criticism and calls for impeachment, he'd already reversed his order, according to court documents obtained by CNN last Friday.

He amended his first ruling, crossing out the line in the order that read, "The Court orders the Division to place the child with a duly married, heterosexual foster-adoptive couple within one week."

Striking out the sentence

Court documents show Johansen wrote initially that it was not in the best interest of children to be raised by same sex couples, citing "belief that research has shown that children are more emotionally and mentally stable when raised by a mother and father in the same home ..."

Johansen, in his amended order, struck the sentence about the best interest of children and scratched out "belief" and replaced it with "concern."

"The judge is clearly reacting to adverse publicity and critical comments regarding his controversial previous ruling," said CNN legal analyst Paul Callan after reviewing the court documents.

Callan said the change suggests that the judge was worried about his order "being viewed as an application of religious belief rather than an application of the law."

'It's not fair, and it's not right'

The judge's initial decision left the parents, April Hoagland and Beckie Peirce heartbroken and shattered.

"It's not fair, and it's not right, and it hurts me very badly, because I have done nothing wrong," Hoagland told CNN affiliate KUTV.

The couple was legally married more than a year ago and approved as foster parents after passing home inspections, background checks and interviews with the division of family services. They want to adopt the infant.

They already are raising Peirce's two children. They say the Utah Division of Child and Family Services, the biological mother and others involved in the case are supportive of them.

The child services division subsequently filed a motion asking the judge to stay his order. If he did not, the agency had said that it would petition a court of appeals.

The case sparked outrage and cries of injustice from rights groups and others.

First challenge

This was the first serious challenge on same-sex adoption rights since a U.S. Supreme Court ruling legalized same-sex marriage this summer, Callan said.

"This is the first time there has been an attempt to deprive gay foster parents of their rights to care for an adoptive child," said Callan.

He said he expects to see a second wave of cases related to same-sex marriages, such as adoption, employment and other issues that the Supreme Court did not address.

"We want to do what's best for the child and make sure she's taken care of in the way she should be," Peirce told Ashleigh Banfield of CNN's "Legal View" before the judge rescinded the ruling. "We don't want her bouncing back and forth."

™ & © 2015 Cable News Network, Inc., a Time Warner Company. All rights reserved.