I have been programming using Python for slightly more than half an year now and I am more interested in Python internals rather than using Python to develop applications. Currently I am working on porting a few libraries from Python2 to Python3. However, I have a rather abstract view on how to make port stuff over from Python2 to Python3 as most of the changes deal with design issues in Python2.x

I&#39;d like to learn more about Python internals; should I go for a top-down or a bottom-up approach? Are there any references you could recommend?

How can I learn more about Python’s internals?

Been browsing through .NET source code of [.NET Framework Reference Source](https://referencesource.microsoft.com/#mscorlib/system/int32.cs,225942ed7b7a3252), just for fun of it. And found something I don&#39;t understand. 

There is a _Int32.cs_ file with C# code for `Int32` type. And somehow that seems strange to me. How does the C# compiler compile code for `Int32` type? 

	public struct Int32: IComparable, IFormattable, IConvertible {
		internal int m_value;

		// ... 
	}

But isn&#39;t this illegal in C#? If `int` is only an _alias for_ `Int32`, it should fail to compile with [Error CS0523](http://msdn.microsoft.com/library/7b625z82.aspx): 

&gt; Struct member &#39;struct2 field&#39; of type &#39;struct1&#39; causes a cycle in the struct layout. 

Is there some magic in the compiler, or am I completely off track?

If Int32 is just an alias for int, how can the Int32 class use an int?

I have 3 projects in my C# solution.

 - Signatures 
 - Structures 
 - Tests

Signatures have public and internal interfaces. 
Also, it has 
 
      [assembly: InternalsVisibleTo(&quot;Structures&quot;)]
      [assembly: InternalsVisibleTo(&quot;Tests&quot;)]

in AssemblyInfo.cs of.

Structures have public and internal classes and

      [assembly: InternalsVisibleTo(&quot;Tests&quot;)]
    
in AssemblyInfo.cs of.

Tests has next source:

    &lt;packages&gt;
      &lt;package id=&quot;Moq&quot; version=&quot;4.2.1409.1722&quot; targetFramework=&quot;net45&quot; /&gt;
      &lt;package id=&quot;NUnit&quot; version=&quot;2.6.4&quot; targetFramework=&quot;net45&quot; /&gt;
      &lt;package id=&quot;NUnitTestAdapter&quot; version=&quot;1.2&quot; targetFramework=&quot;net45&quot; /&gt;
    &lt;/packages&gt;

as NuGet packages in packages.config.

I wrote an unit test for the internal interface from Signatures and internal class from Structures. Run, and had next result: exception:

**Type Signatures.InterfaceX is not visible to DynamicProxy. Can not create proxy for types that are not accessible. Make the type public, or internal and mark your assembly with [assembly: InternalsVisibleTo(InternalsVisible.ToDynamicProxyGenAssembly2)] attribute.**

Seems logical. I added 
```CSharp
[assembly: InternalsVisibleTo(&quot;InternalsVisible.DynamicProxyGenAssembly2&quot;)] 
```
to assembly info of Signatures and Structures projects. Run, and had next result: exception:

**Type &#39;Castle.Proxies.IReminiscenceableDataTableProxy&#39; from assembly &#39;DynamicProxyGenAssembly2, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null&#39; is attempting to implement an inaccessible interface.**

I expected it would help but it didn&#39;t. 
It only changed the exception message.

How to fix my problem?

How to do internal interfaces visible for Moq?

There is a lot of code in one of our projects that looks like this:

    internal static class Extensions
    {
        public static string AddFoo(this string s)
        {
            if (s == null)
            {
                return &quot;Foo&quot;;
            }

            return $({s}Foo);
        }
    }

Is there any explicit reason to do this other than &quot;it is easier to make the type public later?&quot;

I suspect it only matters in very strange edge cases (reflection in Silverlight) or not at all.

Why use a public method in an internal class?

Take the following example:

    int main(void)
    {
         pid_t  pid;
    
         pid = fork();
         if (pid == 0) 
              ChildProcess();
         else 
              ParentProcess();
    }

So correct me if I am wrong, once fork() executes a child process is created. Now going by this [answer][1] fork() returns twice. That is once for the parent process and once for the child process. 

Which means that two separate processes come into existence DURING the fork call and not after it ending. 

Now I don&#39;t get it how it understands how to return 0 for the child process and the correct PID for the parent process. 

This where it gets really confusing. This [answer][2] states that fork() works by copying the context information of the process and manually setting the return value to 0. 

First am I right in saying that the return to any function is placed in a single register? **Since in a single processor environment a process can call only one subroutine that returns only one value (correct me if I am wrong here).** 

**Let&#39;s say I call a function foo() inside a routine and that function returns a value, that value will be stored in a register say BAR. Each time a function wants to return a value it will use a particular processor register.** So if I am able to manually change the return value in the process block I am able to change the value returned to the function right?

***So am I correct in thinking that is how fork() works?***


  [1]: https://stackoverflow.com/a/2562743/5826291
  [2]: https://stackoverflow.com/a/23819945/5826291

How does fork() know when to return 0?

What does the Git index exactly contain, and what command can I use to view the content of the index?

-----

Thanks for all your answers. I know that the index acts as a staging area, and what is committed is in the index rather than the working tree. I am just curious about what an index object consists of. I guess it might be a list of filename/directory names, SHA-1 pairs, a kind of virtual tree maybe?

Is there, in Git terminology, any *plumbing* command that I can use to list the contents of the index?

What does the git index contain EXACTLY?

I keep wondering how does a debugger work? Particulary the one that can be &#39;attached&#39; to already running executable. I understand that compiler translates code to machine language, but then how does debugger &#39;know&#39; what it is being attached to?

How does a debugger work?

Coda Hale&#39;s article [&quot;How To Safely Store a Password&quot;][1] claims that:

&gt; bcrypt has salts built-in to prevent rainbow table attacks.

He cites [this paper][2], which says that in OpenBSD&#39;s implementation of `bcrypt`:

&gt; OpenBSD generates the 128-bit bcrypt salt from an arcfour
&gt; (arc4random(3)) key stream, seeded with random data the kernel
&gt; collects from device timings.

I don&#39;t understand how this can work. In my conception of a salt:

- It needs to be different for each stored password, so that a separate rainbow table would have to be generated for each
- It needs to be stored somewhere so that it&#39;s repeatable: when a user tries to log in, we take their password attempt, repeat the same salt-and-hash procedure we did when we originally stored their password, and compare

When I&#39;m using Devise (a Rails login manager) with bcrypt, there is no salt column in the database, so I&#39;m confused. If the salt is random and not stored anywhere, how can we reliably repeat the hashing process?

In short, **how can bcrypt have built-in salts**?

  [1]: http://codahale.com/how-to-safely-store-a-password/
  [2]: http://www.usenix.org/events/usenix99/provos/provos_html/node6.html#SECTION00051000000000000000

How can bcrypt have built-in salts?

While this question doesn&#39;t have any real use in practice, I am curious as to how Python does string interning. I have noticed the following.

    &gt;&gt;&gt; &quot;string&quot; is &quot;string&quot;
    True

This is as I expected.

You can also do this.

    &gt;&gt;&gt; &quot;strin&quot;+&quot;g&quot; is &quot;string&quot;
    True

And that&#39;s pretty clever!

But you can&#39;t do this.

    &gt;&gt;&gt; s1 = &quot;strin&quot;
    &gt;&gt;&gt; s2 = &quot;string&quot;
    &gt;&gt;&gt; s1+&quot;g&quot; is s2
    False

Why wouldn&#39;t Python evaluate `s1+&quot;g&quot;`, and realize it is the same as `s2` and point it to the same address? What is actually going on in that last block to have it return `False`?


Python string interning

One of the basic data types in R is factors. In my experience factors are basically a pain and I never use them. I always convert to characters. I feel oddly like I&#39;m missing something. 

Are there some important examples of functions that use factors as grouping variables where the factor data type becomes necessary? Are there specific circumstances when I *should* be using factors?

Factors in R: more than an annoyance?

Could you please explain what the practical usage is for the `internal` keyword in C#?

I know that the `internal` modifier limits access to the current assembly, but when and in which circumstance should I use it?

Practical uses for the &quot;internal&quot; keyword in C#

I&#39;ve been working with databases for the last few years and I&#39;d like to think that I&#39;ve gotten fairly competent with using them. However I was reading recently about Joel&#39;s &lt;a href=&quot;http://www.joelonsoftware.com/articles/LeakyAbstractions.html&quot;&gt; Law of Leaky Abstractions&lt;/a&gt; and I realised that even though I can write a query to get pretty much anything I want out of a database, I have no idea how the database actually interprets the query. Does anyone know of any good articles or books that explain how databases work internally?

Some specific things I&#39;m interested in are:
&lt;ul&gt;
&lt;li&gt;What does a database actually do to find out what matches a select statement?&lt;/li&gt;
&lt;li&gt;How does a database interpret a join differently to a query with several &quot;where key1 = key2&quot; statements?&lt;/li&gt;
&lt;li&gt;How does the database store all its memory?&lt;/li&gt;
&lt;li&gt;How are indexes stored?&lt;/li&gt;
&lt;/ul&gt;

How do databases work internally?

Many methods in the .Net library are implemented in native code. Those that come from the framework itself are marked with `[MethodImpl(MethodImplOptions.InternalCall)]`. Those that come from some unmanaged DLL are marked with `[DllImport]` (e.g. `[DllImport(&quot;kernel32.dll&quot;)]`). So far nothing unusual.

But while writing [answer for another question](https://stackoverflow.com/a/9491084/41071), I discovered there are many methods marked with `[DllImport(&quot;QCall&quot;)]`. They seem to be internal implementation of .Net (e.g. `GC._Collect()`).

My question is: What exactly does `[DllImport(&quot;QCall&quot;)]` mean? What is the difference between `[DllImport(&quot;QCall&quot;)]` and `[MethodImpl(MethodImplOptions.InternalCall)]`?

What is [DllImport(&quot;QCall&quot;)]?

One of the key differences between Git and most other version control systems is that the others tend to store commits as a series of deltas - changesets between one commit and the next. This seems logical, since it&#39;s the smallest possible amount of information to store about a commit. But the longer the commit history gets, the more calculation it takes to compare ranges of revisions.

By contrast, Git stores a **complete snapshot of the whole project in each revision**. The reason this doesn&#39;t make the repo size grow dramatically with each commit is each file in the project is stored as a file in the Git subdirectory, named for the hash of its contents. So if the contents haven&#39;t changed, the hash hasn&#39;t changed, and the commit just points to the same file. And there are other optimizations as well.

All this made sense to me until I stumbled on [this information about pack files][1], into which Git puts data periodically to save space:

&gt; In order to save that space, Git
&gt; utilizes the packfile. This is a
&gt; format where Git will only save the
&gt; part that has changed in the second
&gt; file, with a pointer to the file it is
&gt; similar to.

**Isn&#39;t this basically going back to storing deltas? If not, how is it different? How does this avoid subjecting Git to the same problems other version controls systems have?**

For example, Subversion uses deltas, and rolling back 50 versions means undoing 50 diffs, whereas with Git you can just grab the appropriate snapshot. Unless git also stores 50 diffs in the packfiles... is there some mechanism that says &quot;after some small number of deltas, we&#39;ll store a whole new snapshot&quot; so that we don&#39;t pile up too large a changeset? How else might Git avoid the disadvantages of deltas?

  [1]: http://git-scm.com/book/en/v2/Git-Internals-Packfiles

All Internals Solutions on Gang of Coders

Total of 14 Internals Solutions