Voice recognition technology in the South Korean company's Internet connected TVs captures and transmits nearby conversations.
The potential for TVs to eavesdrop is revealed in Samsung's smart TV privacy policy available on its website.
"Please be aware that if your spoken words include personal or other sensitive information, that information will be among the data captured and transmitted to a third party through your use of Voice Recognition," the policy said.
For the voice command feature to work, the TV listens for speech which is translated by third-party software into text and sent back to the TV as a command.
Samsung declined to name the software company. The TV also transmits other information including its unique identifier. Samsung said data collection is aimed at improving TV performance but users can disable it.
In a statement, the company said it takes consumer privacy "very seriously."
"We employ industry-standard security safeguards and practices, including data encryption, to secure consumers' personal information and prevent unauthorised collection or use."
It is not the first time that smart TVs sparked privacy concerns. In 2013, the owner of a LG Electronics smart TV revealed it was sending information about his viewing habits back to the company without consent and without encrypting data.
LG has also experimented with displaying targeted ads on its smart TVs, which requires collecting and utilising user data, such as their location, age and gender.