Multiple instances of singleton across shared libraries on Linux

My question, as the title mentioned, is obvious, and I describe the scenario in details. There is a class named singleton implemented by singleton pattern as following, in file singleton.h:

/*
 * singleton.h
 *
 *  Created on: 2011-12-24
 *      Author: bourneli
 */

#ifndef SINGLETON_H_
#define SINGLETON_H_

class singleton
{
private:
    singleton() {num = -1;}
    static singleton* pInstance;
public:
    static singleton& instance()
    {
        if (NULL == pInstance)
        {
            pInstance = new singleton();
        }
        return *pInstance;
    }
public:
    int num;
};

singleton* singleton::pInstance = NULL;

#endif /* SINGLETON_H_ */

then, there is a plugin called hello.cpp as following:

#include <iostream>
#include "singleton.h"

extern "C" void hello() {
    std::cout << "singleton.num in hello.so : " << singleton::instance().num << std::endl;
    ++singleton::instance().num;
    std::cout << "singleton.num in hello.so after ++ : " << singleton::instance().num << std::endl;
}

you can see that the plugin call the singleton and change the attribute num in the singleton.

last, there is a main function use the singleton and the plugin as following:

#include <iostream>
#include <dlfcn.h>
#include "singleton.h"

int main() {
    using std::cout;
    using std::cerr;
    using std::endl;

    singleton::instance().num = 100; // call singleton
    cout << "singleton.num in main : " << singleton::instance().num << endl;// call singleton

    // open the library
    void* handle = dlopen("./hello.so", RTLD_LAZY);
    
    if (!handle) {
        cerr << "Cannot open library: " << dlerror() << '\n';
        return 1;
    }
    
    // load the symbol
    typedef void (*hello_t)();

    // reset errors
    dlerror();
    hello_t hello = (hello_t) dlsym(handle, "hello");
    const char *dlsym_error = dlerror();
    if (dlsym_error) {
        cerr << "Cannot load symbol 'hello': " << dlerror() << '\n';
        dlclose(handle);
        return 1;
    }
    
    hello(); // call plugin function hello

    cout << "singleton.num in main : " << singleton::instance().num << endl;// call singleton
    dlclose(handle);
}

and the makefile is following:

example1: main.cpp hello.so
    $(CXX) $(CXXFLAGS)  -o example1 main.cpp -ldl

hello.so: hello.cpp
    $(CXX) $(CXXFLAGS)  -shared -o hello.so hello.cpp

clean:
    rm -f example1 hello.so

.PHONY: clean

so, what is the output? I thought there is following:

singleton.num in main : 100
singleton.num in hello.so : 100
singleton.num in hello.so after ++ : 101
singleton.num in main : 101

however, the actual output is following:

singleton.num in main : 100
singleton.num in hello.so : -1
singleton.num in hello.so after ++ : 0
singleton.num in main : 100

It proves that there are two instances of the singleton class.

Why?

First, you should generally use -fPIC flag when building shared libraries.

Not using it "works" on 32-bit Linux, but would fail on 64-bit one with an error similar to:

/usr/bin/ld: /tmp/ccUUrz9c.o: relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC

Second, your program will work as you expect after you add -rdynamic to the link line for the main executable:

singleton.num in main : 100
singleton.num in hello.so : 100
singleton.num in hello.so after ++ : 101
singleton.num in main : 101

In order to understand why -rdynamic is required, you need to know about the way dynamic linker resolves symbols, and about the dynamic symbol table.

First, let's look at the dynamic symbol table for hello.so:

$ nm -C -D hello.so | grep singleton
0000000000000b8c W singleton::instance()
0000000000201068 B singleton::pInstance
0000000000000b78 W singleton::singleton()

This tells us that there are two weak function definitions, and one global variable singleton::pInstance that are visible to the dynamic linker.

Now let's look at the static and dynamic symbol table for the original example1 (linked without -rdynamic):

$ nm -C  example1 | grep singleton
0000000000400d0f t global constructors keyed to singleton::pInstance
0000000000400d38 W singleton::instance()
00000000006022e0 B singleton::pInstance
0000000000400d24 W singleton::singleton()

$ nm -C -D example1 | grep singleton
$ 

That's right: even though the singleton::pInstance is present in the executable as a global variable, that symbol is not present in the dynamic symbol table, and therefore "invisible" to the dynamic linker.

Because the dynamic linker "doesn't know" that example1 already contains a definition of singleton::pInstance, it doesn't bind that variable inside hello.so to the existing definition (which is what you really want).

When we add -rdynamic to the link line:

$ nm -C  example1-rdynamic | grep singleton
0000000000400fdf t global constructors keyed to singleton::pInstance
0000000000401008 W singleton::instance()
00000000006022e0 B singleton::pInstance
0000000000400ff4 W singleton::singleton()

$ nm -C -D  example1-rdynamic | grep singleton
0000000000401008 W singleton::instance()
00000000006022e0 B singleton::pInstance
0000000000400ff4 W singleton::singleton()

Now the definition of singleton::pInstance inside the main executable is visible to the dynamic linker, and so it will "reuse" that definition when loading hello.so:

LD_DEBUG=bindings ./example1-rdynamic |& grep pInstance
     31972:	binding file ./hello.so [0] to ./example1-rdynamic [0]: normal symbol `_ZN9singleton9pInstanceE'

You have to be careful when using runtime-loaded shared libraries. Such a construction is not strictly part of the C++ standard, and you have to consider carefully what the semantics of such a procedure turn out to be.

First off, what's happening is that the shared library sees its own, separate global variable singleton::pInstance. Why is that? A library that's loaded at runtime is essentially a separate, independent program that just happens to not have an entry point. But everything else is really like a separate program, and the dynamic loader will treat it like that, e.g. initialize global variables etc.

The dynamic loader is a runtime facility that has nothing to do with the static loader. The static loader is part of the C++ standard implementation and resolves all of the main program's symbols before the main program starts. The dynamic loader, on the other hand, only runs after the main program has already started. In particular, all symbols of the main program already have to be resolved! There is simply no way to automatically replace symbols from the main program dynamically. Native programs are not "managed" in any way that allows for systematic relinking. (Maybe something can be hacked, but not in a systematic, portable way.)

So the real question is how to solve the design problem that you're attempting. The solution here is to pass handles to all global variables to the plugin functions. Make your main program define the original (and only) copy of the global variable, and the initialize your library with a pointer to that.

For example, your shared library could look like this. First, add a pointer-to-pointer to the singleton class:

class singleton
{
    static singleton * pInstance;
public:
    static singleton ** ppinstance;
    // ...
};

singleton ** singleton::ppInstance(&singleton::pInstance);

Now use *ppInstance instead of pInstance everywhere.

In the plugin, configure the singleton to the pointer from the main program:

void init(singleton ** p)
{
    singleton::ppInsance = p;
}

And the main function, call the plugin intialization:

init_fn init;
hello_fn hello;
*reinterpret_cast<void**>(&init) = dlsym(lib, "init");
*reinterpret_cast<void**>(&hello) = dlsym(lib, "hello");

init(singleton::ppInstance);
hello();

Now the plugin shares the same pointer to the singleton instance as the rest of the program.

I think the simple answer is here: http://www.yolinux.com/TUTORIALS/LibraryArchives-StaticAndDynamic.html

When you have a static variable, it is stored in the object (.o,.a and/or .so)

If the final object to be executed contains two versions of the object, the behaviour is unexpected, like for instance, calling the Destructor of a Singleton object.

Using the proper design, such as declaring the static member in the main file and using the -rdynamic/fpic and using the "" compiler directives will do the trick part for you.

Example makefile statement:

$ g++ -rdynamic -o appexe $(OBJ) $(LINKFLAGS) -Wl,--whole-archive -L./Singleton/ -lsingleton -Wl,--no-whole-archive $(LIBS) 

Hope this works!

Thank you all for your answers!

As a follow-up for Linux, you can also use RTLD_GLOBAL with dlopen(...), per man dlopen (and the examples it has). I've made a variant of the OP's example in this directory: github tree Example output: output.txt

Quick and dirty:

• If you don't want to have to manually link in each symbol to your main, keep the shared objects around. (e.g., if you made *.so objects to import into Python)
• You can initially load into the global symbol table, or do a NOLOAD + GLOBAL re-open.

Code:

#if MODE == 1
// Add to static symbol table.
#include "producer.h"
#endif
...
    #if MODE == 0 || MODE == 1
        handle = dlopen(lib, RTLD_LAZY);
    #elif MODE == 2
        handle = dlopen(lib, RTLD_LAZY | RTLD_GLOBAL);
    #elif MODE == 3
        handle = dlopen(lib, RTLD_LAZY);
        handle = dlopen(lib, RTLD_LAZY | RTLD_NOLOAD | RTLD_GLOBAL);
    #endif

Modes:

• Mode 0: Nominal lazy loading (won't work)
• Mode 1: Include file to add to static symbol table.
• Mode 2: Load initially using RTLD_GLOBAL
• Mode 3: Reload using RTLD_NOLOAD | RTLD_GLOBAL