Save all files in Visual Studio project as UTF-8

I wonder if it's possible to save all files in a Visual Studio 2008 project into a specific character encoding. I got a solution with mixed encodings and I want to make them all the same (UTF-8 with signature).

I know how to save single files, but how about all files in a project?

Since you're already in Visual Studio, why not just simply write the code?

foreach (var f in new DirectoryInfo(@"...").GetFiles("*.cs", SearchOption.AllDirectories)) {
  string s = File.ReadAllText(f.FullName);
  File.WriteAllText (f.FullName, s, Encoding.UTF8);
}

Only three lines of code! I'm sure you can write this in less than a minute :-)

This may be of some help.

> link removed due to original reference being defaced by spam site.

Short version: edit one file, select File -> Advanced Save Options. Instead of changing UTF-8 to Ascii, change it to UTF-8. Edit: Make sure you select the option that says no byte-order-marker (BOM)

Set code page & hit ok. It seems to persist just past the current file.

In case you need to do this in PowerShell, here is my little move:

Function Write-Utf8([string] $path, [string] $filter='*.*')
{
	[IO.SearchOption] $option = [IO.SearchOption]::AllDirectories;
	[String[]] $files = [IO.Directory]::GetFiles((Get-Item $path).FullName, $filter, $option);
	foreach($file in $files)
	{
		"Writing $file...";
		[String]$s = [IO.File]::ReadAllText($file);
		[IO.File]::WriteAllText($file, $s, [Text.Encoding]::UTF8);
	}
}

I would convert the files programmatically (outside VS), e.g. using a Python script:

import glob, codecs

for f in glob.glob("*.py"):
    data = open("f", "rb").read()
    if data.startswith(codecs.BOM_UTF8):
        # Already UTF-8
        continue
    # else assume ANSI code page
    data = data.decode("mbcs")
    data = codecs.BOM_UTF8 + data.encode("utf-8")
    open("f", "wb").write(data)

This assumes all files not in "UTF-8 with signature" are in the ANSI code page - this is the same what VS 2008 apparently also assumes. If you know that some files have yet different encodings, you would have to specify what these encodings are.

Using C#:

1. Create a new ConsoleApplication, then install Mozilla Universal Charset Detector
2. Run code:

static void Main(string[] args)
{
    const string targetEncoding = "utf-8";
    foreach (var f in new DirectoryInfo(@"<your project's path>").GetFiles("*.cs", SearchOption.AllDirectories))
    {
        var fileEnc = GetEncoding(f.FullName);
        if (fileEnc != null && !string.Equals(fileEnc, targetEncoding, StringComparison.OrdinalIgnoreCase))
        {
            var str = File.ReadAllText(f.FullName, Encoding.GetEncoding(fileEnc));
            File.WriteAllText(f.FullName, str, Encoding.GetEncoding(targetEncoding));
        }
    }
    Console.WriteLine("Done.");
    Console.ReadKey();
}

private static string GetEncoding(string filename)
{
    using (var fs = File.OpenRead(filename))
    {
        var cdet = new Ude.CharsetDetector();
        cdet.Feed(fs);
        cdet.DataEnd();
        if (cdet.Charset != null)
            Console.WriteLine("Charset: {0}, confidence: {1} : " + filename, cdet.Charset, cdet.Confidence);
        else
            Console.WriteLine("Detection failed: " + filename);
        return cdet.Charset;
    }
}

Thanks for your solutions, this code has worked for me :

Dim s As String = ""
Dim direc As DirectoryInfo = New DirectoryInfo("Your Directory path")

For Each fi As FileInfo In direc.GetFiles("*.vb", SearchOption.AllDirectories)
	s = File.ReadAllText(fi.FullName, System.Text.Encoding.Default)
	File.WriteAllText(fi.FullName, s, System.Text.Encoding.Unicode)
Next
    

I have created a function to change encoding files written in asp.net. I searched a lot. And I also used some ideas and codes from this page. Thank you.

And here is the function.

  Function ChangeFileEncoding(pPathFolder As String, pExtension As String, pDirOption As IO.SearchOption) As Integer
    
    Dim Counter As Integer
    Dim s As String
    Dim reader As IO.StreamReader
    Dim gEnc As Text.Encoding
    Dim direc As IO.DirectoryInfo = New IO.DirectoryInfo(pPathFolder)
    For Each fi As IO.FileInfo In direc.GetFiles(pExtension, pDirOption)
        s = ""
        reader = New IO.StreamReader(fi.FullName, Text.Encoding.Default, True)
        s = reader.ReadToEnd
        gEnc = reader.CurrentEncoding
        reader.Close()
        
        If (gEnc.EncodingName <> Text.Encoding.UTF8.EncodingName) Then
            s = IO.File.ReadAllText(fi.FullName, gEnc)
            IO.File.WriteAllText(fi.FullName, s, System.Text.Encoding.UTF8)
            Counter += 1
            Response.Write("<br>Saved #" & Counter & ": " & fi.FullName & " - <i>Encoding was: " & gEnc.EncodingName & "</i>")
        End If
    Next

    Return Counter
End Function

It can placed in .aspx file and then called like:

ChangeFileEncoding("C:\temp\test", "*.ascx", IO.SearchOption.TopDirectoryOnly)

if you are using TFS with VS : http://msdn.microsoft.com/en-us/library/1yft8zkw(v=vs.100).aspx Example :

tf checkout -r -type:utf-8 src/*.aspx

If you want to avoid this type of error :

Use this following code :

foreach (var f in new DirectoryInfo(@"....").GetFiles("*.cs", SearchOption.AllDirectories))
            {
                string s = File.ReadAllText(f.FullName, Encoding.GetEncoding(1252));
                File.WriteAllText(f.FullName, s, Encoding.UTF8);
            }

Encoding number 1252 is the default Windows encoding used by Visual Studio to save your files.

Convert from UTF-8-BOM to UTF-8

Building on rasx's answer, here is a PowerShell function that assumes your current files are already encoded in UTF-8 (but maybe with BOM) and converts them to UTF-8 without BOM, therefore preserving existing Unicode characters.

Function Write-Utf8([string] $path, [string] $filter='*')
{
    [IO.SearchOption] $option = [IO.SearchOption]::AllDirectories;
    [String[]] $files = [IO.Directory]::GetFiles((Get-Item $path).FullName, $filter, $option);
    foreach($file in $files)
    {
        "Writing $file...";
        [String]$s = [IO.File]::ReadAllText($file, [Text.Encoding]::UTF8);
        [Text.Encoding]$e = New-Object -TypeName Text.UTF8Encoding -ArgumentList ($false);
        [IO.File]::WriteAllText($file, $s, $e);
    }
}

Experienced encoding problems after converting solution from VS2008 to VS2015. After conversion all project files was encoded in ANSI, but they contained UTF8 content and was recongnized as ANSI files in VS2015. Tried many conversion tactics, but worked only this solution.

 Encoding encoding = Encoding.Default;
 String original = String.Empty;
 foreach (var f in new DirectoryInfo(path).GetFiles("*.cs", SearchOption.AllDirectories))
 {
    using (StreamReader sr = new StreamReader(f.FullName, Encoding.Default))
    {
       original = sr.ReadToEnd();
       encoding = sr.CurrentEncoding;
       sr.Close();
    }
    if (encoding == Encoding.UTF8)
       continue;
    byte[] encBytes = encoding.GetBytes(original);
    byte[] utf8Bytes = Encoding.Convert(encoding, Encoding.UTF8, encBytes);
    var utf8Text = Encoding.UTF8.GetString(utf8Bytes);

    File.WriteAllText(f.FullName, utf8Text, Encoding.UTF8);
 }

the item is removed from the menu in Visual Studio 2017 You can still access the functionality through File-> Save As -> then clicking the down arrow on the Save button and clicking "Save With Encoding...".

You can also add it back to the File menu through Tools->Customize->Commands if you want to.

I'm only offering this suggestion in case there's no way to automatically do this in Visual Studio (I'm not even sure this would work):

1. Create a class in your project named 足の不自由なハッキング (or some other unicode text that will force Visual Studio to encode as UTF-8).
2. Add "using MyProject.足の不自由なハッキング;" to the top of each file. You should be able to do it on everything by doing a global replace of "using System.Text;" with "using System.Text;using MyProject.足の不自由なハッキング;".
3. Save everything. You may get a long string of "Do you want to save X.cs using UTF-8?" messages or something.