President Joe Biden turned the page on one legacy of 9/11 by ending the war in Afghanistan. But he has yet to do much about another: the Guantanamo Bay detention center.

The White House says it intends to shutter the prison on the U.S. base in Cuba, which opened in January 2002 and where most of the 39 men still held have never been charged with a crime. How or when the administration will carry out that plan remains unclear, though early moves to free one prisoner and place five others on a list of those eligible for release have generated optimism among some eager to see it close, including prisoners.

Also Read: White House withdraws Chipman’s ATF chief nomination over Senate opposition

“The fact that Biden, at least, is saying the right things has given people hope,” said Clive Stafford Smith, a lawyer who was recently making his 40th trip to Guantanamo Bay, seeing prisoners he hadn’t been able to visit since the start of the pandemic. “Hope is a dangerous thing because it’s easily crushed. But at the same time, at least, they have hope and that’s good.”

As he did with Afghanistan, Biden faces a complex task in closing Guantanamo. It was a pledge that President Barack Obama famously made, and then failed to carry out. Closure was abandoned as a goal altogether under President Donald Trump, who vowed once to “load it up with some bad dudes” but mostly just ignored the place.

The challenge now, as then, remains: What should the U.S. government do with the some of the men at Guantanamo, including about two dozen it isn’t ready to release?

Among them are Khalid Shaikh Mohammad, a onetime senior al-Qaida figure considered the architect of the 9/11 attacks. He faces a trial by military commission with four co-defendants that — amid legal and logistical challenges, personnel issues and the pandemic — has bogged down in the pretrial stage at a specially built high-security courtroom for more than 9 years. There is no start in sight.

Also Read: US transport authorities to double fines for people refusing to wear masks

Mohammad and his co-defendants were in court this week for the first time since the start of the pandemic for a hearing on the qualifications of a new judge, Air Force Col. Matthew McCall, to preside over the sprawling death penalty case. It was the 42nd session of pretrial hearings since the arraignment in May 2012.

With the passage of time comes new problems. The oldest prisoner, a Pakistani cleared for release in May but who remains at Guantanamo, is 74 and has heart disease and other ailments. A number of other men have significant physical and mental health issues as well that will need to be addressed if “indefinite” detention goes on much longer. Since Guantanamo opened, nine prisoners have died — two from natural causes, and seven in apparent suicides.

“People are getting older, sicker, more and more desperate,” said Pardiss Kebriaei, an attorney with the Center for Constitutional Rights who represents a Yemeni prisoner who was recently cleared but remains held.

It’s not surprising, really, that no one made long-term plans for the detention center. It was a makeshift project from the start.

Following the invasion of Afghanistan, in reaction to the Sept. 11, 2001, attacks, the U.S. wanted a place to hold the hundreds of prisoners from dozens of countries swept up by American forces, many handed over, as it turned out later, in exchange for bounties regardless of whether they had a connection to al-Qaida or the Taliban.

The administration of then President George W. Bush declared they were the “worst of the worst,” and asserted it could hold the men overseas, without charge as unlawful enemy combatants, not entitled to the full protections of prisoners of war at the sleepy Navy outpost on the jagged southeastern coast of Cuba.

A photo released by the Pentagon showed the first detainees, clad in orange jumpsuits, and kneeling in outdoor cages under the tropical sun. It was intended to show a message that “we are doing what we need to do” in a defiant message to the world, said Karen Greenberg, director of the Center on National Security at Fordham Law.

Also Read: US unveils COVID vaccine mandates for companies with 100+ employees

“They regretted that decision very soon afterwards, within days if not weeks,” said Greenberg, author of “The Least Worst Place: Guantanamo’s First 100 Days.”

As reports emerged of brutal treatment, Guantanamo became a source of international outrage, undercutting the sympathy and support the U.S. drew after the 9/11 attacks.

The U.S. would end up holding 779 prisoners at Guantanamo and spend hundreds of millions constructing and operating what today looks more or less like a small state prison, surrounded by razor wire and guard posts at the edge of the shimmering Caribbean Sea.

Bush would ultimately let 532 prisoners out. Obama released 197. Trump released a single detainee, a Saudi who went back to his homeland after striking a plea deal in the problem-plagued military commissions.

Few of those held could be charged with a crime because no evidence was collected when they were captured, or there wasn’t any, or it was tainted beyond use when the detainees were subjected to what the CIA euphemistically called “enhanced interrogation.” Of those who remain, 10 are facing trial by military commission, with all still in the pretrial stage.

Over the years, the population has steadily shrunk as the U.S. decided some men no longer posed a threat and weren’t worth holding amid legal challenges. It has also at times been roiled by hunger strikes and rocked by clashes between prisoners and guards, sparked largely by frustration at being held indefinitely without charge under what the U.S. asserted was its right under the international laws of war.

Guantanamo is smaller and quieter now. But Stafford Smith, a founder of the human rights organization Reprieve, says it’s still oppressive. “It’s not so much the physical conditions, it’s the psychological conditions,” he said. “Being told that you’re in Hotel California and you can check out but you can never leave, that psychologically is immensely damaging to people.”

Obama, who issued an executive order shortly after taking office directing that Guantanamo be closed within a year, ran into political opposition when his administration announced it would move the military trials to federal courts. Congress eventually added language to the annual Pentagon authorization bill prohibiting the government from moving Guantanamo prisoners into the United States for any reason.

In a sign that the political winds might be shifting, Congress recently stripped the prohibition on transferring Guantanamo Bay prisoners from the Pentagon authorization and eliminated funding for the detention center from next year’s budget. It remains to be seen whether that will change, particularly after several former prisoners, released under both Bush and Obama, emerged as Taliban leaders in Afghanistan.

The Biden administration, which didn’t respond to requests for comment for this article, hasn’t said much about its plans.

“I don’t have a timeline for you,” press secretary Jen Psaki told reporters when asked in July about closing Guantanamo. “As you know, there’s a process. There are different layers of the process. But that remains our goal, and we are considering all available avenues to responsibly transfer detainees and, of course, close Guantanamo Bay.”

Those who support closure are encouraged by the fact that the new administration has revived a review board process and has cleared five for release (none were cleared under Trump). But they are concerned that Biden team has yet to name anyone at the State Department to lead an effort to secure agreements with other countries for the resettlement of prisoners, as was done under Obama.

Many argue that the simplest solution would be to move the cases of the 10 detainees facing trial by military commission to federal court in the U.S. and find a way to transfer or release the rest. Kebriaei, the attorney whose Yemeni client is awaiting release, said the administration just needs to focus on the issue.

“There’s a sense that it has to be done and very practically more of a possibility that it can be done,” she said.