WATCH: Obama Tearfully Says Goodbye to Attorney General Eric Holder

In a heartfelt and emotional ceremony Friday, President Obama began to formally say goodbye to attorney general Eric Holder, calling him "one of our finest."

The outgoing attorney general's formal portrait was also unveiled during what the White House calls a "surprise" performance of "America The Beautiful" by Aretha Franklin.

Amid commendations for the "hundreds of terrorism convictions" and "long-overdue reform of our criminal justice division," the President highlighted Holder's dedication to restoring "what he calls the conscience of the nation -- our Civil Rights Division."

In addition to Holder's unwavering defense of voting rights throughout the country, Obama pointed to Holder's leadership within the administration to securing greater equality for same-sex couples.

"Several years ago, Eric recommended that our government stop defending the Defense of Marriage Act, because he wants our country to be a place where love is love," said Obama. "And where same-sex marriage is recognized on the federal level, and same-sex couples can receive the same federal benefits as anybody else."

Closing his remarks, the president appeared to become emotional, wiping tears from his eyes as he spoke of Holder's virtue.

"Having good men in positions of power of authority who are willing to fight for what's right -- that's a rare thing," said Obama "That's a powerful thing. It's something that shapes our future in ways we don't even understand."

In his tenure at the Justice Department, Holder, the first African-American to hold the post of attorney general, has been integral to several key legal moves to advance LGBT equality. Holder had previously served as U.S. attorney for the District of Columbia and deputy U.S. attorney general under Janet Reno during the Clinton administration. Prior to joining the Obama administration, he was a civil rights attorney who testified in 1999 House Judiciary Committee to advocate for stronger hate-crimes protections.

Holder, who is the third-longest-serving attorney general in the nation's history, offered a similarly heartfelt speech. While he reflected on the accomplishments achieved by his Department of Justice and the Obama Administration in general, he stressed that there is more work to be done.

"Great as it is, our nation is not yet perfect," said Holder. "The fact that we can acknowledge this is what truly distinguishes us as a people. We have always examined ourselves and determined that which needs to be improved, that which needs to be maintained, and that to which we should aspire. This is the essence of and the beauty of the United States of America. Unlike other countries, complacent in an older, sclerotic system, we are still young, dynamic, and unafraid to question ourselves.

"This spirit initially led to revolution, and then to the removal of the sin of slavery, the right of women to vote, a great Civil Rights Movement that truly transformed our nation, and now a recognition of the rights of all Americans regardless of their sexual orientation," continued Holder.

Holder will likely be replaced by Obama's nominee for attorney general, Loretta Lynch. Lynch earned approval from a key Senate Judiciary Committee this week, all but guaranteeing her final confirmation by the full Senate in the coming weeks.

Watch Obama and Holder's remarks below.