[Linux]: How do I remove ‘Control M’ i.e., (^M) characters from a file
To create a file with “Control M” (^M) characters – you can type Ctrl+V,Ctrl+M from keyboard through vi editor in a linux file.
And to view “Control M” characters in a file, you can use “cat -v filename” or “cat -A filename” ; you can not usually see them with normal cat command.
[Dev root @ dbversitydotcom /tmp]# cat -v dbversity.txt
^M
^M ^M
==============
Amazon AuroraDB
PostgreSQL
mariaDB ^M
==============
============================
Oracle & MS-SQL
MongoDB^M
============================
^M^M^M
[Dev root @ dbversitydotcom /tmp]
tr command to remove Control M characters.
[Dev root @ dbversitydotcom /tmp]# tr -d ‘\r’ < dbversity.txt > new_dbversity.txt
[Dev root @ dbversitydotcom /tmp]#
[Dev root @ dbversitydotcom /tmp]# mv new_dbversity.txt dbversity.txt
mv: overwrite `dbversity.txt’? y
[Dev root @ dbversitydotcom /tmp]#
[Dev root @ dbversitydotcom /tmp]# rm new_dbversity.txt
[Dev root @ dbversitydotcom /tmp]# cat -v dbversity.txt
==============
Amazon AuroraDB
PostgreSQL
mariaDB
==============
============================
Oracle & MS-SQL
MongoDB
============================
[Dev root @ dbversitydotcom /tmp]
Another solution : dos2unix command will fix it.
dos2unix filename
Reason for ^M characters in a file :
Windows-based text editors put special characters at the end of lines to denote a line return or newline. Normally harmless, some applications on a Linux server cannot understand these characters and can cause the service to not respond correctly. This can result in unforeseen complications. This is because the file is created or perhaps even edited on a machine running Windows and then uploaded to a Linux server.
Example : (rsa) public/private keys
—–BEGIN CERTIFICATE—–^M
ZXIIDBjCAe4CCQD3cWtEoYMtujANBgkqhkiG9w0BAQUFADBFMQswCQYDVQQGEwJB^M
VTETMBEGA1UECBMKU29tZS1TdGF0ZTEhMB8GA1UEChMYSW50ZXJuZXQgV2lkZ2l0^M
cyBQdHkgTHRkMB4XDTEzMDcxNzIxMTU0NFoXDTE1MDcxNzIxMTU0NFowRTELMAkG^M
A1UEBhMCQVUxEzARBgNVBAgTClNvbWUtU3RhdGUxITAfBgNVBAoTGEludGVybmV0^M
IFdpZGdpdHMgUHR5IEx0ZDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB^M
ANIQ4wo6zi9lY5WddpJbHO2OvXgbJZIB8qidksKrFm3tUMe6LiXpOb4fZAwqlUUp^M
fN+jrp+Lr7PPF52vnbKMP4dq31CWV68EtcRK3O8maLy4mpx7pCXeZIqRw3/1UwIr^M
RXzOCWTsdvudAC0Sp0DgN8U7qECojZso9nu1SBopOtBDDU/Wcdmd3ydZJN0R7waB^M
e38WNK6rUNCgcIP0obvuiwppJmUyE1hr09A3W2zGB6bMF4EqZ5uaiBoPhMyKTlDs^M
UMOd6tRXkrYEdnLuqQttUvtE6sfxZaUTmod5MCXSRrW0tAkBGhqlPBT8NvlRQzaz^M
TqzLqhCYPQ9HBihumKHRkucCAwEAATANBgkqhkiG9w0BAQUFAAOCAQEALAGKt8Bn^M
PHYPj6RNioE8eyCvx4WFG8zDIkbkFbmpBiq6SOpWliCnhGWyH5yFEDWHCOebz8L9^M
LHQTi18KDGk7PH0BpWZDpksu3JrBb3fjm+1QCpD9KWRYJVnBGAaWFFmPPpJJDoz2^M
X0MRPbzxfqsno68l46X+5l5m4H8Y2g0cOUto580x/ZlNb+gHbTzuBnwGa2Oof0bc^M
A7ZmU1IytADXWVM28h88XcBooT4dwxCVkwznNGQ8lQ+XTehjdcDEEvbTB/Jc+1M4^M
0a/OvmlDIFU34DKH/LRc7bmTe4YTTRtU5ShXTBH0ChZFuqP20vtiEsPHVn+pk4LN^M
129c5uhrxPGhNQ==^M
—–END CERTIFICATE—–^M
The file command is also a useful way to check files with ^M (aka “ASCII text, with CRLF line terminators”), for example:
$ ps > plain.txt
$ man zcat > zcat-man-page.txt
$ file *.txt
copied-from-windows.txt: ASCII text, with CRLF line terminators
plain.txt: ASCII text
zcat-man-page.txt: UTF-8 Unicode text, with overstriking
The file “copied-from-windows.txt” was created using notepad.exe or wordpad.exe on Windows. Strange behavior can happen on Linux/Unix when files have ^M in them. For example, here’s what happens if you run a shell script created using notepad.exe or wordpad.exe…
$ cat hello.sh
#!/bin/bash
echo Hello World
exit 0
$ ./hello.sh
-bash: ./hello.sh: /bin/bash^M: bad interpreter: No such file or directory
D’oh!…
$ file hello.sh
hello.sh: Bourne-Again shell script, ASCII text executable, with CRLF line terminators