The 70-year-old's contract expires at the end of June and Cricket Australia directors will consider changes to the selection committee at a board meeting this week.
"The man most likely to step into the role would be Rod Marsh, who has served alongside him (Inverarity) since 2011, when the panel was revamped in the wake of the Argus review," Fairfax Media said.
Former Test batsman Mark Waugh could also be a significant addition as a selector should a proposal from CA team performance manager Pat Howard be ratified.
The only two members of the current panel to have their positions as selectors already confirmed by the board are Marsh and coach Darren Lehmann, a leading Australian daily said.
Part of Australia's golden era in the 1970s, Marsh played 96 Tests during a 14-year career and has a significant post-playing pedigree for the chairman's job.
He headed up Australia's national academy for more than a decade and then took up the same position in England in 2001 before also becoming an England selector.
Marsh, 66, stepped down from the England academy and as a selector in 2005 after playing a key role in England's first Ashes victory for 16 years.
He became an Australian selector in November 2011 as part of a new panel following the Argus review, which recommended sweeping changes after Australia's 2010-11 Ashes debacle against England.


Latest News from Sports News Desk