The NFL star said he and Swift “can’t listen” to the “outside noise” during a press conference on Friday

Travis Kelce says he and Taylor Swift are shutting out the “outside noise” as they navigate their relationship in the public eye.

On Friday, Kelce, 34, answered questions about his rising fame during a Chiefs press conference and shared that he and Swift, 34, have discussed the public attention on their romance.

In response to a reporter who specifically asked Kelce if the topic has come up with Swift, the tight end said he and the “Karma” singer “talked about” managing privacy. “As long as we’re happy, we can’t listen to anything that’s outside noise,” Kelce said. “Thats all that matters.”

ALBERTO E. RODRIGUEZ/GETTY IMAGES; KEVIN MAZUR/GETTY IMAGES

Kelce said he’d “be silly” not to acknowledge the increased attention on him over the last year. “I think the Super Bowl helped that, and obviously how I live my off-field life, definitely helps out,” he said, referring to the infamous 2023 “Kelce Bowl” matchup between his Chiefs and his brother, Jason Kelce’s Philadelphia Eagles, which his side won.

The first-ever brother-on-brother matchup in the Super Bowl earned Kelce a hosting gig on Saturday Night Live and his mother, Donna Kelce, became a beloved personality by NFL fans at the time.

And amid his relationship with Swift, Kelce and his family have garnered an entirely new, and sizable, fanbase of Swifties who adore the 12-time Grammy-winning artist.

Taylor Swift, Travis Kelce.DAVID EULITT/GETTY IMAGES; DAVID GRAY/KANSAS CITY CHIEFS VIA AP

“I brought this upon myself, and I do enjoy having fun with it all, and the biggest thing is making sure my focus is right here in this building,” Kelce told reporters during Friday’s presser.

Kelce continued to point out all of the “outside noise” from the media and fans, and said it’s challenging “dealing with everyone else’s perspective of things” as he gets criticism for having a very public romance during a busy NFL season.

Kansas City Chiefs tight end Travis Kelce (87) flexes to celebrate a 4-yard touchdown reception in the second quarter of an NFL football game between the Philadelphia Eagles and Kansas City Chiefs on Nov 20, 2023.SCOTT WINTERS/ICON SPORTSWIRE VIA GETTY IMAGES

“You hear the media throughout the year if we’re not having success, you know, maybe throw it out there that I wasn’t focused or that the team isn’t focused on certain things, and if you’re in this building, you know exactly what’s going on,” said Kelce.

“So, you gotta compartmentalize what you’re hearing and make sure that you’re staying on task, and giving everybody in the building and on that team the right perception.”

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

Kelce and Swift have been linked since last September when the singer attended her first Chiefs game at Arrowhead Stadium. She’s been a regular supporter of the NFL star throughout the season, and hangs out with his mother Donna as they cheer him on.

On Sunday, Kelce and the Chiefs will compete against the Baltimore Ravens in the AFC Championship for a spot in Super Bowl LVIII, which takes place on Feb. 11 in Las Vegas.