With a global turnover of nearly USD 100 billion, IBM is a major player in India and employs a large workforce in the country.
Nearly one-third of its global workforce of over four lakh persons is in India, where its headcount exceeds even that in US.
The meeting between Modi and Rometty was part of a series of one-on-one meets the Prime Minister had here with top business leaders of US on his last day in New York.
The Prime Minister is also likely to meet business leaders in Washington during the second leg of his five-day visit to US.
During her meeting with Modi, Romnetty discussed "software for Smart Cities and Digital India initiatives," External Affairs Ministry spokesperson Syed Akbaruddin tweeted.
Among other places, IBM has its facilities in 14 Indian cities, including Bangalore, Delhi, Mumbai, Chennai and Hyderabad.
Other business leaders who had one-on-one meetings with the Prime Minister included KKR CEO Henry Kravis and CEOs of BlackRock, GE and Goldman Sachs.
Earlier, Modi also had a breakfast meeting with CEOs of 11 large American corporations.
These included PepsiCo CEO Indra Nooyi, Google Chairman Eric Schmidt, Citigroup chief Michael Corbat, Mastercard CEO Ajay Banga, Cargill's President and CEO David W MacLennan, Caterpillar's Douglas Oberhelman, AES' Andres Gluski, Merck's Kenneth Frazier, Co-founder and Co-CEO of Carlyle Group David Rubenstein, Hospira's Michael Ball and Warburg Pincus' Charles Kaye.