Unfortunately, some programs can take a long time to compile. A couple of hours of compilation is not strange for tensorflow on your setup.

There are reports of it taking 50 minutes [on a considerably faster machine](https://gist.github.com/Brainiarc7/6d6c3f23ea057775b72c52817759b25c#gistcomment-2621082)

A solution to this problem is to use pre-compiled binaries that are available with pip, instructions can be found here: https://www.tensorflow.org/install/pip.html

Basically you can do this:

    pip install tensorflow

If you require a specific older version, like 1.15, you can do this:

    pip install tensorflow==1.15

For gpu support you add `-gpu` to the package name, like this:

    pip install tensorflow-gpu

And:

    pip install tensorflow-gpu==1.15



I&#39;m playing around a bit with the new C# 8 nullable reference types feature, and while refactoring my code I came upon this (simplified) method:

```
public T Get&lt;T&gt;(string key)
{
	var wrapper = cacheService.Get(key);
    return wrapper.HasValue ? Deserialize&lt;T&gt;(wrapper) : default;
}
```

Now, this gives a warning 

&gt; Possible null reference return

 which is logical, since `default(T)` will give null for all reference types. At first I thought I would change it to the following:

`public T? Get&lt;T&gt;(string key)`

But this cannot be done. It says I either have to add a generic constraint `where T : class` or `where T : struct`. But that is not an option, as it can be both (I can store an `int` or `int?` or an instance of `FooBar` or whatever in the cache).
I also read about a supposed new generic constraint `where class?` but that did not seem to work.

The only simple solution I can think of is changing the return statement using a *null forgiving operator*:

    return wrapper.HasValue ? Deserialize&lt;T&gt;(wrapper) : default!;

But that feels wrong, since it can definitely be null, so I&#39;m basically lying to the compiler here..

How can I fix this? Am I missing something utterly obvious here?

Nullable reference types with generic return type

Reference: https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8186315

I&#39;m really struggling to find out what MinRAMPercentage does, especially compared to InitialRAMPercentage.

I assumed that InitialRAMPercentage sets the amount of heap at startup, that MinRAMPercentage and MaxRAMPercentage set the bottom and top limit of heap that the JVM is allowed to shrink/grow to.

Apparently that is not the case. When I start a JVM (with UseContainerSupport, having these new memory setting parameters) like so:

```
java -XX:+UseContainerSupport -XX:InitialRAMPercentage=40.0 -XX:MinRAMPercentage=20.0 -XX:MaxRAMPercentage=80.0 -XX:+PrintFlagsFinal -version | grep Heap
```
InitialHeap and MaxHeap get set, there is no &quot;Minimum Heap Size&quot; value that I can find; Consequently, that MinRAMPercentage never seems to get used.

Super confused, and apparently, I&#39;m not the only one; the OpenJ9 dudes seem to also not fully parse the intent of these options, as I&#39;ve gathered [here](https://github.com/eclipse/openj9/issues/1428) and [here](https://github.com/eclipse/openj9/pull/1699). They seem to have opted to simply not implement MinRAMPercentage afaics.

So: What is the real intended usage and effect of setting MinRAMPercentage?

Clarification of meaning new JVM memory parameters InitialRAMPercentage and MinRAMPercentage

So I am trying to compile `TensorFlow` from the source (using a clone from their git repo from 2019-01-31). I installed Bazel from their shell script (https://github.com/bazelbuild/bazel/releases/download/0.21.0/bazel-0.21.0-installer-linux-x86_64.sh).

I executed `./configure` in the tensorflow code and provided the default settings except for adding my machine specific -m options (-mavx2 -mfma) and pointing python to the correct python3 location (/usr/bin/py3). I then ran the following command as per the tensorflow instructions:

    bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package //tensorflow:libtensorflow_framework.so //tensorflow:libtensorflow.so

Now that continues to run and run, I haven&#39;t seen it complete yet (though I am limited to letting it run for a maximum of about 10 hours). It produces a ton of INFO: warnings regarding signed and unsigned integers and control reaching the end of non-void functions. None of these appear fatal. Compilation continues to tick with the two numbers continuing to grow (&#39;[N,NNN / X,XXX] 4 actions running&#39;) and files ticking through &#39;Compiling&#39;.

The machine is an EC2 instance with ~16GiB of RAM, CPU is &#39;Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz&#39; with I believe 4-cores, plenty of HDD space (although compilation seems to eat QUITE a bit, &gt; 1GiB)

Any ideas on what&#39;s going on here?

Tensorflow Compile Runs For A Long Time

So I am trying to compile <code>TensorFlow</code> from the source (using a clone from their git repo from 2019-01-31). I installed Bazel from their shell script (<a href="https://github.com/bazelbuild/bazel/releases/download/0.21.0/bazel-0.21.0-installer-linux-x86_64.sh" target="_blank" rel="noopener noreferrer">https://github.com/bazelbuild/bazel/releases/download/0.21.0/bazel-0.21.0-installer-linux-x86_64.sh</a>).
I executed <code>./configure</code> in the tensorflow code and provided the default settings except for adding my machine specific -m options (-mavx2 -mfma) and pointing python to the correct python3 location (/usr/bin/py3). I then ran the following command as per the tensorflow instructions:
<pre><code class="hljs language-bash">bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package //tensorflow:libtensorflow_framework.so //tensorflow:libtensorflow.so
</code></pre>
Now that continues to run and run, I haven't seen it complete yet (though I am limited to letting it run for a maximum of about 10 hours). It produces a ton of INFO: warnings regarding signed and unsigned integers and control reaching the end of non-void functions. None of these appear fatal. Compilation continues to tick with the two numbers continuing to grow ('[N,NNN / X,XXX] 4 actions running') and files ticking through 'Compiling'.
The machine is an EC2 instance with ~16GiB of RAM, CPU is 'Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz' with I believe 4-cores, plenty of HDD space (although compilation seems to eat QUITE a bit, > 1GiB)
Any ideas on what's going on here?

So I just came across what seems to me like a strange Python feature and wanted some clarification about it.  

The following array manipulation somewhat makes sense:

    p = [1,2,3]
    p[3:] = [4] 
    p = [1,2,3,4]

I imagine it is actually just appending this value to the end, correct?  
Why can I do this, however?  

    p[20:22] = [5,6]
    p = [1,2,3,4,5,6]
   
And even more so this:  

    p[20:100] = [7,8]
    p = [1,2,3,4,5,6,7,8]

This just seems like wrong logic. It seems like this should throw an error! 
 
 Any explanation?  
-Is it just a weird thing Python does?  
-Is there a purpose to it?  
-Or am I thinking about this the wrong way?


Why does Python allow out-of-range slice indexes for sequences?

I&#39;ve installed Python 3.7 on my Ubuntu 18.04 machine. Following this instructions in case it&#39;s relevant: 

&gt; Download : Python 3.7 from Python Website [1] ,on Desktop and manually
&gt; unzip it, on Desktop Installation : Open Terminal (ctrl +shift+T)
&gt; 
&gt;     Go to the Extracted folder
&gt;     $ cd ~/Desktop/Python-3.7.0
&gt;     $ ./configure
&gt;     $ make
&gt;     $ sudo make install
&gt; 
&gt; Making Python 3.7 default Python :
&gt; 
&gt;     $ sudo vim ~/.bashrc
&gt;     press i
&gt;     on the last and new line - Type
&gt;     alias python= python3.7
&gt;     press Esc
&gt;     type - to save and exit vim
&gt;     :wq
&gt;     now type
&gt;     $ source ~/.bashrc
From here: https://www.quora.com/How-can-I-upgrade-Python-3-6-to-3-7-in-Ubuntu-18-04

I&#39;ve downloaded several modules through `pip install module` but when I try to import them, I get a `ModuleNotFoundError: No module names &#39;xx&#39;`

So I did some research and apparently when used pip to install, it installed in the modules in previous version of Python. 
Somewhere (probably a question in SO) I found a suggestion to install the module using `python3.7 -m pip install module` but then I get `/usr/local/bin/python3.7: no module named pip`.

Now I&#39;m stuck, pip is installed, but apparently not for Python 3.7. I&#39;m assuming that if I can install pip for Python 3.7, I can run the pip install command and get the modules I need.
If that is the case, how can I install pip for python 3.7, since it&#39;s already installed?

---

This is the best I have come up with:

I have installed python 3.7 successfully and I can install modules using pip (or pip3) but those modules are installed in Python 3.6 (Comes with ubuntu). Therefore I can&#39;t import those modules in python 3.7 (get a module not found)

Python 3.7 doesn&#39;t recognize pip/pip3, so I can&#39;t install through pip/pip3
I need python 3.7


How to Install pip for python 3.7 on Ubuntu 18?

I want to get feature names after I fit the pipeline.

    categorical_features = [&#39;brand&#39;, &#39;category_name&#39;, &#39;sub_category&#39;]
    categorical_transformer = Pipeline(steps=[
        (&#39;imputer&#39;, SimpleImputer(strategy=&#39;constant&#39;, fill_value=&#39;missing&#39;)),
        (&#39;onehot&#39;, OneHotEncoder(handle_unknown=&#39;ignore&#39;))])
        
    numeric_features = [&#39;num1&#39;, &#39;num2&#39;, &#39;num3&#39;, &#39;num4&#39;]
    numeric_transformer = Pipeline(steps=[
        (&#39;imputer&#39;, SimpleImputer(strategy=&#39;median&#39;)),
        (&#39;scaler&#39;, StandardScaler())])

    preprocessor = ColumnTransformer(
        transformers=[
            (&#39;num&#39;, numeric_transformer, numeric_features),
            (&#39;cat&#39;, categorical_transformer, categorical_features)])
Then

    clf = Pipeline(steps=[(&#39;preprocessor&#39;, preprocessor),
                          (&#39;regressor&#39;, GradientBoostingRegressor())])

After fitting with pandas dataframe, I can get feature importances from 

`clf.steps[1][1].feature_importances_`

and I tried `clf.steps[0][1].get_feature_names()` but I got an error 

    AttributeError: Transformer num (type Pipeline) does not provide get_feature_names.

How can I get feature names from this?


Sklearn Pipeline: Get feature names after OneHotEncode In ColumnTransformer

When I tried to read a pickle file that saved by a former version of pandas, it yielded an `ImportError`.

&gt; ImportError: No module named &#39;pandas.core.internals.managers&#39;;
&gt; &#39;pandas.core.internals&#39; is not a package

There was no hit on stackoverflow so i would like to share my solution for this particular problem.

ImportError: No module named &#39;pandas.core.internals.managers&#39;; &#39;pandas.core.internals&#39; is not a package

What is the correct way to perform gradient clipping in pytorch?

I have an exploding gradients problem.

How to do gradient clipping in pytorch?

I have installed Ancaconda3 and Tensorflow. When I try to import Tensorflow in python shell I receive the following error:

&gt; ModuleNotFoundError: No module named &#39;numpy.core._multiarray_umath&#39;
&gt; ImportError: numpy.core.multiarray failed to import
&gt; 
&gt; The above exception was the direct cause of the following exception:
&gt; 
&gt; Traceback (most recent call last):   File &quot;&lt;frozen
&gt; importlib._bootstrap&gt;&quot;, line 980, in _find_and_load SystemError:
&gt; &lt;class &#39;_frozen_importlib._ModuleLockManager&#39;&gt; returned a result with
&gt; an error set ImportError: numpy.core._multiarray_umath failed to
&gt; import ImportError: numpy.core.umath failed to import

I am not sure what the problem is as numpy is installed on my system and can be successfully imported in python.
I am using Windows10.

Thanks in advance.



When importing tensorflow, I get the following error: No module named &#39;numpy.core._multiarray_umath&#39;

When I am executing the command `sess = tf.Session()` in Tensorflow 2.0 environment, I am getting an error message as below:

    Traceback (most recent call last):
    File &quot;&lt;stdin&gt;&quot;, line 1, in &lt;module&gt;
    AttributeError: module &#39;tensorflow&#39; has no attribute &#39;Session&#39;

&gt; System Information:

 - OS Platform and Distribution: Windows 10
 - Python Version: 3.7.1
 - Tensorflow Version: 2.0.0-alpha0 (installed with pip)

&gt; Steps to reproduce:
&gt;
&gt; Installation:

 1. pip install --upgrade pip
 2. **pip install tensorflow==2.0.0-alpha0**
 3. pip install keras
 4. pip install numpy==1.16.2

&gt; Execution:

 1. Execute command: import tensorflow as tf
 2. Execute command: sess = tf.Session()

Tensorflow 2.0 - AttributeError: module &#39;tensorflow&#39; has no attribute &#39;Session&#39;

I have installed Cuda 10.1 and cudnn on Ubuntu 18.04 and it seems to be installed properly as type nvcc and nvidia-smi, I get proper response:

    user:~$ nvcc -V
    nvcc: NVIDIA (R) Cuda compiler driver
    Copyright (c) 2005-2019 NVIDIA Corporation
    Built on Fri_Feb__8_19:08:17_PST_2019
    Cuda compilation tools, release 10.1, V10.1.105
    user:~$ nvidia-smi 
    Mon Mar 18 14:36:47 2019       
    +-----------------------------------------------------------------------------+
    | NVIDIA-SMI 418.43       Driver Version: 418.43       CUDA Version: 10.1     |
    |-------------------------------+----------------------+----------------------+
    | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
    | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
    |===============================+======================+======================|
    |   0  Quadro K5200        Off  | 00000000:03:00.0  On |                  Off |
    | 26%   39C    P8    14W / 150W |    225MiB /  8118MiB |      0%      Default |
    +-------------------------------+----------------------+----------------------+
                                                                                   
    +-----------------------------------------------------------------------------+
    | Processes:                                                       GPU Memory |
    |  GPU       PID   Type   Process name                             Usage      |
    |=============================================================================|
    |    0      1538      G   /usr/lib/xorg/Xorg                            32MiB |
    |    0      1583      G   /usr/bin/gnome-shell                           5MiB |
    |    0      3008      G   /usr/lib/xorg/Xorg                           100MiB |
    |    0      3120      G   /usr/bin/gnome-shell                          82MiB |
    +-----------------------------------------------------------------------------+


I have installed tensorflow using:
`user:~$ sudo pip3 install --upgrade tensorflow-gpu`

```
The directory &#39;/home/amin/.cache/pip/http&#39; or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo&#39;s -H flag.
The directory &#39;/home/amin/.cache/pip&#39; or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo&#39;s -H flag.
Requirement already up-to-date: tensorflow-gpu in /usr/local/lib/python3.6/dist-packages (1.13.1)
Requirement already satisfied, skipping upgrade: keras-applications&gt;=1.0.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (1.0.7)
Requirement already satisfied, skipping upgrade: protobuf&gt;=3.6.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (3.6.1)
Requirement already satisfied, skipping upgrade: wheel&gt;=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (0.32.3)
Requirement already satisfied, skipping upgrade: absl-py&gt;=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (0.7.0)
Requirement already satisfied, skipping upgrade: keras-preprocessing&gt;=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (1.0.9)
Requirement already satisfied, skipping upgrade: gast&gt;=0.2.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (0.2.2)
Requirement already satisfied, skipping upgrade: termcolor&gt;=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (1.1.0)
Requirement already satisfied, skipping upgrade: grpcio&gt;=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (1.18.0)
Requirement already satisfied, skipping upgrade: tensorflow-estimator&lt;1.14.0rc0,&gt;=1.13.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (1.13.0)
Requirement already satisfied, skipping upgrade: six&gt;=1.10.0 in /usr/lib/python3/dist-packages (from tensorflow-gpu) (1.11.0)
Requirement already satisfied, skipping upgrade: numpy&gt;=1.13.3 in /usr/lib/python3/dist-packages (from tensorflow-gpu) (1.13.3)
Requirement already satisfied, skipping upgrade: astor&gt;=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (0.7.1)
Requirement already satisfied, skipping upgrade: tensorboard&lt;1.14.0,&gt;=1.13.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-gpu) (1.13.1)
Requirement already satisfied, skipping upgrade: h5py in /usr/local/lib/python3.6/dist-packages (from keras-applications&gt;=1.0.6-&gt;tensorflow-gpu) (2.9.0)
Requirement already satisfied, skipping upgrade: setuptools in /usr/local/lib/python3.6/dist-packages (from protobuf&gt;=3.6.1-&gt;tensorflow-gpu) (40.6.3)
Requirement already satisfied, skipping upgrade: mock&gt;=2.0.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-estimator&lt;1.14.0rc0,&gt;=1.13.0-&gt;tensorflow-gpu) (2.0.0)
Requirement already satisfied, skipping upgrade: werkzeug&gt;=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tensorboard&lt;1.14.0,&gt;=1.13.0-&gt;tensorflow-gpu) (0.14.1)
Requirement already satisfied, skipping upgrade: markdown&gt;=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard&lt;1.14.0,&gt;=1.13.0-&gt;tensorflow-gpu) (3.0.1)
Requirement already satisfied, skipping upgrade: pbr&gt;=0.11 in /usr/local/lib/python3.6/dist-packages (from mock&gt;=2.0.0-&gt;tensorflow-estimator&lt;1.14.0rc0,&gt;=1.13.0-&gt;tensorflow-gpu) (5.1.1)
```

However when I am trying to import tensorflow I am getting error about  libcublas.so.10.0:


    user:~$ python3
    Python 3.6.7 (default, Oct 22 2018, 11:32:17) 
    [GCC 8.2.0] on linux
    Type &quot;help&quot;, &quot;copyright&quot;, &quot;credits&quot; or &quot;license&quot; for more information.
    &gt;&gt;&gt; import tensorflow as tf
    Traceback (most recent call last):
      File &quot;/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow.py&quot;, line 58, in &lt;module&gt;
        from tensorflow.python.pywrap_tensorflow_internal import *
      File &quot;/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py&quot;, line 28, in &lt;module&gt;
        _pywrap_tensorflow_internal = swig_import_helper()
      File &quot;/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py&quot;, line 24, in swig_import_helper
        _mod = imp.load_module(&#39;_pywrap_tensorflow_internal&#39;, fp, pathname, description)
      File &quot;/usr/lib/python3.6/imp.py&quot;, line 243, in load_module
        return load_dynamic(name, filename, file)
      File &quot;/usr/lib/python3.6/imp.py&quot;, line 343, in load_dynamic
        return _load(spec)
    ImportError: libcublas.so.10.0: cannot open shared object file: No such file or directory
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File &quot;&lt;stdin&gt;&quot;, line 1, in &lt;module&gt;
      File &quot;/usr/local/lib/python3.6/dist-packages/tensorflow/__init__.py&quot;, line 24, in &lt;module&gt;
        from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
      File &quot;/usr/local/lib/python3.6/dist-packages/tensorflow/python/__init__.py&quot;, line 49, in &lt;module&gt;
        from tensorflow.python import pywrap_tensorflow
      File &quot;/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow.py&quot;, line 74, in &lt;module&gt;
        raise ImportError(msg)
    ImportError: Traceback (most recent call last):
      File &quot;/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow.py&quot;, line 58, in &lt;module&gt;
        from tensorflow.python.pywrap_tensorflow_internal import *
      File &quot;/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py&quot;, line 28, in &lt;module&gt;
        _pywrap_tensorflow_internal = swig_import_helper()
      File &quot;/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py&quot;, line 24, in swig_import_helper
        _mod = imp.load_module(&#39;_pywrap_tensorflow_internal&#39;, fp, pathname, description)
      File &quot;/usr/lib/python3.6/imp.py&quot;, line 243, in load_module
        return load_dynamic(name, filename, file)
      File &quot;/usr/lib/python3.6/imp.py&quot;, line 343, in load_dynamic
        return _load(spec)
    ImportError: libcublas.so.10.0: cannot open shared object file: No such file or directory
    
    
    Failed to load the native TensorFlow runtime.
    
    See https://www.tensorflow.org/install/errors
    
    for some common reasons and solutions.  Include the entire stack trace
    above this error message when asking for help.


What I am missing? and How can I resolve this?

Thanks

ImportError: libcublas.so.10.0: cannot open shared object file: No such file or directory

I am trying to run some code to create an LSTM model but i get an error:

```AttributeError: module &#39;tensorflow&#39; has no attribute &#39;get_default_graph&#39;```




My code is as follows:

```
from keras.models import Sequential

model = Sequential()
model.add(Dense(32, input_dim=784))
model.add(Activation(&#39;relu&#39;))
model.add(LSTM(17))
model.add(Dense(1, activation=&#39;sigmoid&#39;))
model.compile(loss=&#39;binary_crossentropy&#39;, optimizer=&#39;adam&#39;, metrics=[&#39;accuracy&#39;])
```

I have found someone else with a similar problem and they updated tensorflow and it works; but mine is up to date and still does not work. I am new to using keras and machine learning so I apologise if this is something silly!

How to fix &quot;AttributeError: module &#39;tensorflow&#39; has no attribute &#39;get_default_graph&#39;&quot;?

I am applying transfer-learning on a pre-trained network using the GPU version of keras. I don&#39;t understand how to define the parameters **`max_queue_size`**, **`workers`**, and **`use_multiprocessing`**. If I change these parameters (primarily to speed-up learning), I am unsure whether all data is still seen per epoch.

**`max_queue_size`**:
    
 + maximum size of the internal training queue which is used to &quot;precache&quot; samples from the generator 

 + *Question:* Does this refer to how many batches are prepared on CPU? How is it related to `workers`? How to define it optimally?


**`workers`**: 

 + number of threads generating batches in parallel. Batches are computed in parallel on the CPU and passed on the fly onto the GPU for neural network computations 

 + *Question:* How do I find out how many batches my CPU can/should generate in parallel?


**`use_multiprocessing`**: 

 + whether to use process-based threading

 + *Question:* Do I have to set this parameter to true if I change `workers`? Does it relate to CPU usage?

**Related questions** can be found here:

 + [Detailed explanation of model.fit_generator() parameters: queue size, workers and use_multiprocessing][1]
 + [https://stackoverflow.com/questions/51790943/what-does-worker-mean-in-fit-generator-in-keras][2]

 + [What is the parameter “max_q_size” used for in “model.fit_generator”?](https://stackoverflow.com/questions/36986815/what-is-the-parameter-max-q-size-used-for-in-model-fit-generator/36989864#36989864)

 + [A detailed example of how to use data generators with Keras](https://stanford.edu/~shervine/blog/keras-how-to-generate-data-on-the-fly).

I am using `fit_generator()` as follows:
```
    history = model.fit_generator(generator=trainGenerator,
                                  steps_per_epoch=trainGenerator.samples//nBatches,     # total number of steps (batches of samples)
                                  epochs=nEpochs,                   # number of epochs to train the model
                                  verbose=2,                        # verbosity mode. 0 = silent, 1 = progress bar, 2 = one line per epoch
                                  callbacks=callback,               # keras.callbacks.Callback instances to apply during training
                                  validation_data=valGenerator,     # generator or tuple on which to evaluate the loss and any model metrics at the end of each epoch
                                  validation_steps=
                                  valGenerator.samples//nBatches,   # number of steps (batches of samples) to yield from validation_data generator before stopping at the end of every epoch
                                  class_weight=classWeights,                # optional dictionary mapping class indices (integers) to a weight (float) value, used for weighting the loss function
                                  max_queue_size=10,                # maximum size for the generator queue
                                  workers=1,                        # maximum number of processes to spin up when using process-based threading
                                  use_multiprocessing=False,        # whether to use process-based threading
                                  shuffle=True,                     # whether to shuffle the order of the batches at the beginning of each epoch
                                  initial_epoch=0)   
```


The specs of my machine are:

    CPU : 2xXeon E5-2260 2.6 GHz
    Cores: 10
    Graphic card: Titan X, Maxwell, GM200
    RAM: 128 GB
    HDD: 4TB
    SSD: 512 GB


  [1]: https://github.com/keras-team/keras/issues/8540
  [2]: https://stackoverflow.com/questions/51790943/what-does-worker-mean-in-fit-generator-in-keras

How to define max_queue_size, workers and use_multiprocessing in keras fit_generator()?

I&#39;ve had a super frustrating bug in my code for the past few weeks. My code would work exactly as expected on my computer, but as soon as I farmed it out to the HPC server, it would produce weird results.

I&#39;ve boiled it down to this: on my computer (iMac) the function `abs()` works with floating point numbers, but on the server `abs()` truncates it to an integer.

Example:

Server

    abs(-1.1341234) = 1

My Mac

    abs(-1.1341234) = 1.1341234

Now I know I can fix this by using `fabs()`, that&#39;s not the question. I looked at the versions of `gcc` on either machine and this is the output:

Server

    g++ -v
    Using built-in specs.
    COLLECT_GCC=g++
    COLLECT_LTO_WRAPPER=/apps/software/GCCcore/5.4.0/libexec/gcc/x86_64-unknown-linux-gnu/5.4.0/lto-wrapper
    Target: x86_64-unknown-linux-gnu
    Configured with: ../configure --enable-languages=c,c++,fortran --enable-lto --enable-checking=release --disable-multilib --enable-shared=yes --enable-static=yes --enable-threads=posix --enable-gold=default --enable-plugins --enable-ld --with-plugin-ld=ld.gold --prefix=/apps/software/GCCcore/5.4.0 --with-local-prefix=/apps/software/GCCcore/5.4.0 --enable-bootstrap --with-isl=/dev/shm/GCCcore/5.4.0/dummy-/gcc-5.4.0/stage2_stuff
    Thread model: posix
    gcc version 5.4.0 (GCC) 

My Mac

    g++ -v
    Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/c++/4.2.1
    Apple LLVM version 10.0.1 (clang-1001.0.46.3)
    Target: x86_64-apple-darwin18.5.0
    Thread model: posix
    InstalledDir: /Library/Developer/CommandLineTools/usr/bin

So my question, why does `abs()` produce different results between gcc and clang? This issue literally cost me 3 weeks of progress, so as you can imagine I&#39;m a little salty just now...

Why is `abs()` implemented differently?

My question is about `InterruptedException`, which is thrown from the `Thread.sleep` method. While working with `ExecutorService` I noticed some weird behaviour that I don&#39;t understand; here is what I mean:

    ExecutorService executor = Executors.newSingleThreadExecutor();
        executor.submit(() -&gt; {
            while(true)
            {
                //DO SOMETHING
                Thread.sleep(5000);
            }
        });

With this code, the compiler doesn&#39;t give me any error or message that `InterruptedException` from `Thread.sleep` should be caught. But when I am trying to change the loop condition and replace &quot;true&quot; with some variable like this:

    ExecutorService executor = Executors.newSingleThreadExecutor();
        executor.submit(() -&gt; {
            while(tasksObserving)
            {
                //DO SOMETHING
                Thread.sleep(5000);
            }
        });

The compiler constantly complains that `InterruptedException` has to be handled. Can someone explain to me why this happens, and why if the condition is set to true the compiler ignores the InterruptedException?

Thread.sleep inside infinite while loop in lambda doesn&#39;t require &#39;catch (InterruptedException)&#39; - why not?

I am curious why the following piece of code:

    #include &lt;string&gt;
    int main()
    {
        std::string a = &quot;ABCDEFGHIJKLMNO&quot;;
    }

when compiled with `-O3` yields the following code:

    main:                                   # @main
        xor     eax, eax
        ret

(I perfectly understand that there is no need for the unused `a` so the compiler can entirely omit it from the generated code)

However the following program:

    #include &lt;string&gt;
    int main()
    {
        std::string a = &quot;ABCDEFGHIJKLMNOP&quot;; // &lt;-- !!! One Extra P 
    }

yields:

	main:                                   # @main
	        push    rbx
	        sub     rsp, 48
	        lea     rbx, [rsp + 32]
	        mov     qword ptr [rsp + 16], rbx
	        mov     qword ptr [rsp + 8], 16
	        lea     rdi, [rsp + 16]
	        lea     rsi, [rsp + 8]
	        xor     edx, edx
	        call    std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;::_M_create(unsigned long&amp;, unsigned long)
	        mov     qword ptr [rsp + 16], rax
	        mov     rcx, qword ptr [rsp + 8]
	        mov     qword ptr [rsp + 32], rcx
	        movups  xmm0, xmmword ptr [rip + .L.str]
	        movups  xmmword ptr [rax], xmm0
	        mov     qword ptr [rsp + 24], rcx
	        mov     rax, qword ptr [rsp + 16]
	        mov     byte ptr [rax + rcx], 0
	        mov     rdi, qword ptr [rsp + 16]
	        cmp     rdi, rbx
	        je      .LBB0_3
	        call    operator delete(void*)
	.LBB0_3:
	        xor     eax, eax
	        add     rsp, 48
	        pop     rbx
	        ret
	        mov     rdi, rax
	        call    _Unwind_Resume
	.L.str:
	        .asciz  &quot;ABCDEFGHIJKLMNOP&quot;

when compiled with the same `-O3`. I don&#39;t understand why it does not recognize that the `a` is still unused, regardless that the string is one byte longer.

This question is relevant to gcc 9.1 and clang 8.0, (online: https://gcc.godbolt.org/z/p1Z8Ns) because other compilers in my observation either entirely drop the unused variable (ellcc) or generate code for it regardless the length of the string.


Compiler stops optimizing unused string away when adding characters

This is more of a theoretical question than anything. I&#39;m a Comp sci major with a huge interest in low level programming. I love finding out how things work under the hood. My specialization is compiler design.

Anyway, as I&#39;m working on my first compiler, things are occurring to me that are kind of confusing.

When you write a program in C/C++, the traditional thing people know is, a compiler magically turns your C/C++ code into native code for that machine.

But something doesn&#39;t add up here. If I compile my C/C++ program targeting the x86 architecture, it would seem that the same program should run on any computer with the same architecture. But that doesn&#39;t happen. You need to recompile your code for OS X or Linux or Windows.(And yet again for 32-bit vs 64-bit)

I&#39;m just wondering why this is the case? Don&#39;t we target the CPU architecture/instruction set when compiling a C/C++ program? And a Mac OS and a Windows Os can very much be running on the same exact architecture.

(I know Java and similar target a VM or CLR so those don&#39;t count)

If I took a best-shot answer at this, I&#39;d say C/C++ must then compile to OS-specific instructions. But every source I read says the compiler targets the machine. So I&#39;m very confused.

Content Type	Original Author	Original Content on Stackoverflow
Question	Zonyl	View Question on Stackoverflow
Solution 1 - Python	Leafcloud - jopdorp	View Answer on Stackoverflow

Tensorflow Compile Runs For A Long Time

Python Problem Overview

Python Solutions

Solution 1 - Python

Clarification of meaning new JVM memory parameters InitialRAMPercentage and MinRAMPercentage

Nullable reference types with generic return type

Attributions