In striking down the proposed question, the court held that Ross's justification for including it was a sham.
"It follows that a court cannot sustain agency action founded on a pretextual or sham justification that conceals the true 'basis' for the decision," U.S. District Court Judge Jesse Furman wrote.
The Commerce Department has said that the changing explanations supplement, rather than contradict, Ross's original version of events.
The Trump administration is expected to appeal the decision to the U.S. 2nd Circuit Court of Appeals, and ultimately the battle could make its way to the U.S. Supreme Court. The Justice Department said it was still reviewing the ruling, but was disappointed in the outcome.
"Secretary Ross, the only person with legal authority over the census, reasonably decided to reinstate a citizenship question on the 2020 census in response to the Department of Justice's request for better citizenship data, to protect voters against racial discrimination," spokeswoman Kelly Laco said in a statement.
"Not only has the government asked a citizenship question in the census for most of the last 200 years, 41 million households have already answered it on the American Community Survey since 2005. Our government is legally entitled to include a citizenship question on the census and people in the United States have a legal obligation to answer," she said.
The high court in November agreed to review whether Ross could be deposed about his reasons for adding the citizenship question. It was not immediately clear what the effect of Tuesday's decision would be on arguments in that case scheduled to take place next month.
Ruling from Manhattan, Furman agreed with the plaintiffs in the case, which included civil rights groups, 18 states and 15 cities. He said that adding the question to the census would likely prevent possibly millions of people from being counted, primarily among those who live in households containing noncitizens and Hispanic people.
"That undercount, in turn, will translate into a loss of political power and funds, among other harms," Furman wrote.
In a statement, New York Attorney General Letitia James praised the ruling.
"The attempts by the Trump Administration to mandate a question about citizenship were not rooted in a desire to strengthen the census process and would only undermine our immigrant communities. Inciting fear in our residents is not only immoral, but also ill-conceived," she said.