Although the ruling applies only to death certificates, his statements about the ban were sweeping and unequivocal, and are expected to incite further litigation challenging the law. Ohio's attorney general said the state will appeal.
Black cited the Supreme Court's June decision striking down part of a federal anti-gay marriage law, saying the lower courts are now tasked with applying that ruling.
"And the question presented is whether a state can do what the federal government cannot — i.e., discriminate against same-sex couples ... simply because the majority of the voters don't like homosexuality (or at least didn't in 2004)," Black said in reference to the year Ohio's gay marriage ban passed. "Under the Constitution of the United States, the answer is no."
His ruling stems from a July lawsuit by two gay Ohio men whose spouses recently died and wanted to be recognized as married on their death certificates.
The men's attorney, Al Gerhardstein, said he was considering whether to file further litigation right away or let Black's decision "percolate for a little bit."
"I can tell you that the reasoning in this opinion is broad and the principals he set out are firmly rooted in solid legal arguments, but they would support a broader attempt on marriage recognition and marriage celebration in Ohio, so we're looking at that," he said.
Attorney General Mike DeWine said the state will appeal Black's decision to the 6th U.S. Circuit Court of Appeals, based in Cincinnati. He called Monday's decision "not a huge surprise," given earlier rulings Black made in the case.
"Our job is to defend the Ohio Constitution and state statutes ... and that's what we intend to do," DeWine said.
Bridget Coontz, the attorney who argued on behalf of the state, said Wednesday in Black's Cincinnati courtroom that in the Supreme Court's historic June decision, the justices also found that states have the right to decide for themselves whether to recognize gay marriage, and Ohio voters decided not to in 2004.
"Ohio doesn't want Delaware or Maryland to define who is married under Ohio law," she said. "To allow that to happen would allow one state to set the marriage policy for all others."
Black said constitutional rights trump Ohio's gay marriage ban, questioning whether it was passed for a legitimate state interest "other than simply maintaining a 'traditional' definition of marriage."
He quoted then-Gov. Robert Taft, who said in 2004 that the law was intended "to reaffirm existing Ohio law with respect to our most basic, rooted, and time-honored institution: marriage between a man and a woman."
Black wrote that "the fact that a form of discrimination has been 'traditional' is a reason to be more skeptical of its rationality."
"No hypothetical justification can overcome the clear primary purpose and practical effect of the marriage bans ... to disparage and demean the dignity of same-sex couples in the eyes of the state and the wider community," Black wrote.
Eighteen states and the District of Columbia allow same-sex weddings, up from six before the Supreme Court's decision in June.
Also Monday, a federal judge in Utah allowed gay weddings to continue there, rejecting a request to put them on hold as the state appeals a decision that sent couples flocking to county clerks for marriage licenses.
Judge Robert Shelby overturned Utah's ban on same-sex marriage on Friday, ruling the voter-approved measure is a violation of gay couples' constitutional rights.
New Mexico's highest court also legalized gay marriage Thursday.