The **pickle** module implements an algorithm for turning an arbitrary **Python** object into a series of bytes. This process is also called serializing” the object. The byte stream representing the object can then be transmitted or stored, and later reconstructed to create a new object with the same characteristics.

The **cPickle** module implements the same algorithm, in **C** instead of Python. It is many times faster than the Python implementation, but does not allow the user to subclass from Pickle. If subclassing is not important for your use, you probably want to use cPickle.

[Source](https://pymotw.com/2/pickle/) of above information.

I found this regarding pickle and cPickle:

&quot;The pickle module implements an algorithm for turning an arbitrary Python object into a series of bytes....

The cPickle module implements the same algorithm, in C instead of Python. It is **many times faster** than the Python implementation, **but does not allow the user to subclass from Pickle.** 

If subclassing is not important for your use, you probably want to use cPickle.&quot;


Source: https://pymotw.com/2/pickle/ 

I&#39;m working on an universal app with *all orientations* on iPad and *only portrait* on iPhone. The app works well with split-screen multitasking on iOS 9 compatible iPad, but I have this warning:

    All interface orientations must be supported unless the app requires full screen

And my app does not require full screen. It&#39;s only limited to portrait on iPhone... Shouldn&#39;t it be ok? Is there a way to declare *Requires Full Screen* only on iPhone?

Thanks in advance

By the way I&#39;m using Xcode 7.3.1

iOS 9 : Warning &quot;All interface orientations must be supported unless the app requires full screen&quot; for universal app

I am trying to connect to RDS Database from an AWS Lambda (Java).

Which IP should I enable from the RDS Security group rules?

Allow AWS Lambda to access RDS Database

Python 2 has both the `pickle` and `cPickle` modules for serialization.

`cPickle` has an obvious advantage over `pickle`: speed. What, if any, advantage does `pickle` have over `cPickle`?

Python2: Should I use Pickle or cPickle?

<p>Python 2 has both the <code>pickle</code> and <code>cPickle</code> modules for serialization.</p>
<p><code>cPickle</code> has an obvious advantage over <code>pickle</code>: speed. What, if any, advantage does <code>pickle</code> have over <code>cPickle</code>?</p>


I installed Anaconda for Python 2.7 on my Windows machine and wanted to add the Anaconda interpreter to PyDev, but quick googling couldn&#39;t find the default place where Anaconda installed, and searching SO didn&#39;t turn up anything useful, so.

Where does Anaconda 4.0 install on Windows 7?

Where does Anaconda Python install on Windows?

I have a Pandas DataFrame with a column containing lists objects

          A
    0   [1,2]
    1   [3,4]
    2   [8,9] 
    3   [2,6]

How can I access the first element of each list and save it into a new column of the DataFrame? To get a result like this:

          A     new_col
    0   [1,2]      1
    1   [3,4]      3
    2   [8,9]      8
    3   [2,6]      2

I know this could be done via iterating over each row, but is there any &quot;pythonic&quot; way? 

Accessing every 1st element of Pandas DataFrame column containing lists

This might be silly but I am unable to install `cPickle` with python 3.5 docker image

**Dockerfile**

    FROM python:3.5-onbuild

**requirements.txt**

    cpickle

When I try to build the image

    $ docker build -t sample .
    Sending build context to Docker daemon 3.072 kB
    Step 1 : FROM python:3.5-onbuild
    # Executing 3 build triggers...
    Step 1 : COPY requirements.txt /usr/src/app/
    Step 1 : RUN pip install --no-cache-dir -r requirements.txt
     ---&gt; Running in 016c35a032ee
    Collecting cpickle (from -r requirements.txt (line 1))
      Could not find a version that satisfies the requirement cpickle (from -r requirements.txt (line 1)) (from versions: )
    No matching distribution found for cpickle (from -r requirements.txt (line 1))
    You are using pip version 7.1.2, however version 8.1.1 is available.
    You should consider upgrading via the &#39;pip install --upgrade pip&#39; command.
    The command &#39;/bin/sh -c pip install --no-cache-dir -r requirements.txt&#39; returned a non-zero code: 1



installing cPickle with python 3.5

I am using Python 3.5.1. I read the document and the package section here: https://docs.python.org/3/tutorial/modules.html#packages

Now, I have the following structure:

    /home/wujek/Playground/a/b/module.py

`module.py`:

    class Foo:
        def __init__(self):
            print(&#39;initializing Foo&#39;)

Now, while in `/home/wujek/Playground`:

    ~/Playground $ python3
    &gt;&gt;&gt; import a.b.module
    &gt;&gt;&gt; a.b.module.Foo()
    initializing Foo
    &lt;a.b.module.Foo object at 0x100a8f0b8&gt;

Similarly, now in home, superfolder of `Playground`:

    ~ $ PYTHONPATH=Playground python3
    &gt;&gt;&gt; import a.b.module
    &gt;&gt;&gt; a.b.module.Foo()
    initializing Foo
    &lt;a.b.module.Foo object at 0x10a5fee10&gt;

Actually, I can do all kinds of stuff:

    ~ $ PYTHONPATH=Playground python3
    &gt;&gt;&gt; import a
    &gt;&gt;&gt; import a.b
    &gt;&gt;&gt; import Playground.a.b

Why does this work? I though there needed to be `__init__.py` files (empty ones would work) in both `a` and `b` for `module.py` to be importable when the Python path points to the `Playground` folder?

This seems to have changed from Python 2.7:

    ~ $ PYTHONPATH=Playground python
    &gt;&gt;&gt; import a
    ImportError: No module named a
    &gt;&gt;&gt; import a.b
    ImportError: No module named a.b
    &gt;&gt;&gt; import a.b.module
    ImportError: No module named a.b.module

With `__init__.py` in both `~/Playground/a` and `~/Playground/a/b` it works fine.

Is __init__.py not required for packages in Python 3.3+

I&#39;m trying to mock a singluar method from the boto3 s3 client object to throw an exception. But I need all other methods for this class to work as normal.

This is so I can test a singular Exception test when and error occurs performing a [upload_part_copy][1]

**1st Attempt**

    import boto3
    from mock import patch

    with patch(&#39;botocore.client.S3.upload_part_copy&#39;, side_effect=Exception(&#39;Error Uploading&#39;)) as mock:
        client = boto3.client(&#39;s3&#39;)
        # Should return actual result
        o = client.get_object(Bucket=&#39;my-bucket&#39;, Key=&#39;my-key&#39;)
        # Should return mocked exception
        e = client.upload_part_copy()

However this gives the following error:

    ImportError: No module named S3

**2nd Attempt**

After looking at the botocore.client.py source code I found that it is doing something clever and the method `upload_part_copy` does not exist. I found that it seems to call `BaseClient._make_api_call` instead so I tried to mock that

    import boto3
    from mock import patch

    with patch(&#39;botocore.client.BaseClient._make_api_call&#39;, side_effect=Exception(&#39;Error Uploading&#39;)) as mock:
        client = boto3.client(&#39;s3&#39;)
        # Should return actual result
        o = client.get_object(Bucket=&#39;my-bucket&#39;, Key=&#39;my-key&#39;)
        # Should return mocked exception
        e = client.upload_part_copy()

This throws an exception... but on the `get_object` which I want to avoid.

Any ideas about how I can only throw the exception on the `upload_part_copy` method?

  [1]: http://boto3.readthedocs.io/en/latest/reference/services/s3.html#S3.Client.upload_part_copy

Mocking boto3 S3 client method Python

I&#39;m writing a script to automate data generation for a demo and I need to serialize in a JSON some data. Part of this data is an image, so I encoded it in base64, but when I try to run my script I get:

    Traceback (most recent call last):
      File &quot;lazyAutomationScript.py&quot;, line 113, in &lt;module&gt;
        json.dump(out_dict, outfile)
      File &quot;/usr/lib/python3.4/json/__init__.py&quot;, line 178, in dump
        for chunk in iterable:
      File &quot;/usr/lib/python3.4/json/encoder.py&quot;, line 422, in _iterencode
        yield from _iterencode_dict(o, _current_indent_level)
      File &quot;/usr/lib/python3.4/json/encoder.py&quot;, line 396, in _iterencode_dict
        yield from chunks
      File &quot;/usr/lib/python3.4/json/encoder.py&quot;, line 396, in _iterencode_dict
        yield from chunks
      File &quot;/usr/lib/python3.4/json/encoder.py&quot;, line 429, in _iterencode
        o = _default(o)
      File &quot;/usr/lib/python3.4/json/encoder.py&quot;, line 173, in default
        raise TypeError(repr(o) + &quot; is not JSON serializable&quot;)
      TypeError: b&#39;iVBORw0KGgoAAAANSUhEUgAADWcAABRACAYAAABf7ZytAAAABGdB...
         ...
       BF2jhLaJNmRwAAAAAElFTkSuQmCC&#39; is not JSON serializable

As far as I know, a base64-encoded-whatever (a PNG image, in this case) is just a string, so it should pose to problem to serializating. What am I missing?

Serialize in JSON a base64 encoded data

Is `json.loads` from Python&#39;s standard `json` module vulnerable to arbitrary code execution or any other security problems?

My application can receive JSON messages from non-trustworthy sources.

Is json.loads() vulnerable to arbitrary code execution?

I want to clarify the given documentation of Django-rest-framework regarding the creation of a model object. So far I have found that there are 3 approaches on how to handle such events.

 1. The Serializer&#39;s `create()` method. Here is the [documentation][1]

        class CommentSerializer(serializers.Serializer):
    
            def create(self, validated_data):
                return Comment.objects.create(**validated_data)
    

 2. The ModelViewset `create()` method. [Documentation][2]

        class AccountViewSet(viewsets.ModelViewSet):
    
            queryset = Account.objects.all()
            serializer_class = AccountSerializer
            permission_classes = [IsAccountAdminOrReadOnly]

 3. The ModelViewset `perform_create()` method. [Documentation][3]

        class SnippetViewSet(viewsets.ModelViewSet):

            def perform_create(self, serializer):
                serializer.save(owner=self.request.user)


These three approaches are important depending on your application environment. 
But when do we need to use each `create() / perform_create()` function? On the other hand, I found some accounts that two create methods were called for a single post request the `ModelViewSet`&#39;s `create()` and serializer&#39;s `create()`.


  [1]: http://www.django-rest-framework.org/api-guide/serializers/#saving-instances
  [2]: http://www.django-rest-framework.org/api-guide/viewsets/#modelviewset
  [3]: http://www.django-rest-framework.org/tutorial/6-viewsets-and-routers/#refactoring-to-use-viewsets

When to use Serializer&#39;s create() and ModelViewset&#39;s perform_create()

I wonder what&#39;s the correct way of converting (deserializing) a string to a Python&#39;s Enum class. Seems like `getattr(YourEnumType, str)` does the job, but I&#39;m not sure if it&#39;s safe enough.

Just to be more specific, I would like to convert a `&#39;debug&#39;`string to an Enum object like this:

    class BuildType(Enum):
        debug = 200
        release = 400



Convert string to Enum in Python

I was looking for alternative ways to save a trained model in PyTorch. So far, I have found two alternatives.

1. [torch.save()][1] to save a model and [torch.load()][2] to load a model.
2. [model.state_dict()][3] to save a trained model and [model.load_state_dict()][4] to load the saved model.

I have come across to this [discussion][5] where approach 2 is recommended over approach 1.

My question is, why the second approach is preferred? Is it only because [torch.nn][6] modules have those two function and we are encouraged to use them?


  [1]: https://github.com/torch/torch7/blob/master/doc/serialization.md#torchsavefilename-object--format-referenced
  [2]: https://github.com/torch/torch7/blob/master/doc/serialization.md#object-torchloadfilename--format-referenced
  [3]: http://pytorch.org/docs/nn.html#torch.nn.Module.state_dict
  [4]: http://pytorch.org/docs/nn.html#torch.nn.Module.load_state_dict
  [5]: https://discuss.pytorch.org/t/how-to-save-load-torch-models/718
  [6]: http://pytorch.org/docs/nn.html

Best way to save a trained model in PyTorch?

At times, I&#39;ve seen ``.pickle``, ``.pck``, ``.pcl``, and ``.db`` for files that contain Python pickles, but I am unsure what is the most common or best practice. I know that the latter three extensions are also used for other things.

The related question is: What MIME type is preferred for sending pickles between systems using a REST API?


Preferred (or most common) file extension for a Python pickle

When I tried to read a pickle file that saved by a former version of pandas, it yielded an `ImportError`.

&gt; ImportError: No module named &#39;pandas.core.internals.managers&#39;;
&gt; &#39;pandas.core.internals&#39; is not a package

There was no hit on stackoverflow so i would like to share my solution for this particular problem.

ImportError: No module named &#39;pandas.core.internals.managers&#39;; &#39;pandas.core.internals&#39; is not a package

I&#39;m trying to restore a pickled config file from RLLib ([json didn&#39;t work as shown in this post][1]), and getting the following error:

```
config = pickle.load(open(f&quot;{path}/params.pkl&quot;, &quot;rb&quot;))

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
&lt;ipython-input-28-c964561b863c&gt; in &lt;module&gt;
----&gt; 1 config = pickle.load(open(f&quot;{path}/params.pkl&quot;, &quot;rb&quot;))

ValueError: unsupported pickle protocol: 5

```

Python Version = 3.7.0

How can I open this file in 3.7?


  [1]: https://stackoverflow.com/questions/62908522/error-callbacks-must-be-a-callable-method-that-returns-a-subclass-of-defaultc

Python 3.7 Error: Unsupported Pickle Protocol 5

I want to print some floating point numbers so that they&#39;re always written in decimal form (e.g. `12345000000000000000000.0` or `0.000000000000012345`, not in [scientific notation](https://en.wikipedia.org/wiki/Scientific_notation), yet I&#39;d want to the result to have the up to ~15.7 [significant figures](https://en.wikipedia.org/wiki/Significant_figures) of a IEEE 754 double, and no more. 

What I want is ***ideally* so that the result is the *shortest* string in positional decimal format that still results in the same value when converted to a `float`**.

It is well-known that the `repr` of a `float` is written in scientific notation if the exponent is greater than 15, or less than -4:
 
    &gt;&gt;&gt; n = 0.000000054321654321
    &gt;&gt;&gt; n
    5.4321654321e-08  # scientific notation

If `str` is used, the resulting string again is in scientific notation:

    &gt;&gt;&gt; str(n)
    &#39;5.4321654321e-08&#39;

----

It has been suggested that I can use `format` with `f` flag and sufficient precision to get rid of the scientific notation:

    &gt;&gt;&gt; format(0.00000005, &#39;.20f&#39;)
    &#39;0.00000005000000000000&#39;

It works for that number, though it has some extra trailing zeroes. But then the same format fails for `.1`, which gives decimal digits beyond the actual machine precision of float:

    &gt;&gt;&gt; format(0.1, &#39;.20f&#39;)
    &#39;0.10000000000000000555&#39;

And if my number is `4.5678e-20`, using `.20f` would still lose relative precision:

    &gt;&gt;&gt; format(4.5678e-20, &#39;.20f&#39;)
    &#39;0.00000000000000000005&#39;

Thus **these approaches do not match my requirements**.

----

This leads to the question: what is the easiest and also well-performing way to print arbitrary floating point number in decimal format, having the same digits as in [`repr(n)` (or `str(n)` on Python 3)](https://stackoverflow.com/a/28493269/918959), but always using the decimal format, not the scientific notation.

That is, a function or operation that for example converts the float value `0.00000005` to string `&#39;0.00000005&#39;`; `0.1` to `&#39;0.1&#39;`; `420000000000000000.0` to `&#39;420000000000000000.0&#39;` or `420000000000000000` and formats the float value `-4.5678e-5` as `&#39;-0.000045678&#39;`.

----

After the bounty period: It seems that there are at least 2 viable approaches, as Karin demonstrated that using string manipulation one can achieve significant speed boost compared to my initial algorithm on Python 2.

Thus,

 - If performance is important and Python 2 compatibility is required; or if the `decimal` module cannot be used for some reason, then [Karin&#39;s approach using string manipulation](https://stackoverflow.com/a/38983595/918959) is the way to do it.
 - On Python 3, [my somewhat shorter code will also be faster](https://stackoverflow.com/a/38847691/918959).

Since I am primarily developing on Python 3, I will accept my own answer, and shall award Karin the bounty.


Convert float to string in positional format (without scientific notation and false precision)

I&#39;m getting this error

&gt; TypeError: super() takes at least 1 argument (0 given)

using this code on python2.7.11: 

    class Foo(object):
        def __init__(self):
            pass

    class Bar(Foo):
        def __init__(self):
            super().__init__()

    Bar()


The workaround to make it work would be:

    class Foo(object):
        def __init__(self):
            pass

    class Bar(Foo):
        def __init__(self):
            super(Bar, self).__init__()

    Bar()


It seems the syntax is specific to python 3. So, what&#39;s the best way to provide compatible code between 2.x and 3.x and avoiding this error happening?

TypeError: super() takes at least 1 argument (0 given) error is specific to any python version?

Why does the `set` function call wipe out the dupes, but parsing a set literal does not?

    &gt;&gt;&gt; x = Decimal(&#39;0&#39;)
    &gt;&gt;&gt; y = complex(0,0)
    &gt;&gt;&gt; set([0, x, y])
    {0}
    &gt;&gt;&gt; {0, x, y}
    {Decimal(&#39;0&#39;), 0j}

(Python 2.7.12.  Possibly same root cause as for [this][1] similar question)


  [1]: https://stackoverflow.com/q/40225520/674039

Set literal gives different result from set function call

I recently came across a syntax I never seen before when I learned python nor in most tutorials, the `..` notation, it looks something like this:

    f = 1..__truediv__ # or 1..__div__ for python 2
    
    print(f(8)) # prints 0.125 

I figured it was exactly the same as (except it&#39;s longer, of course):

    f = lambda x: (1).__truediv__(x)
    print(f(8)) # prints 0.125 or 1//8

But my questions are: 

* How can it do that?  
* What does it actually mean with the two dots? 
* How can you use it in a more complex statement (if possible)? 

This will probably save me many lines of code in the future...:)

What is `1..__truediv__` ? Does Python have a .. (&quot;dot dot&quot;) notation syntax?

I have been having issues with openssl and python@2 with brew, [which have explained here (unresolved)](https://stackoverflow.com/questions/60229970/aws-cli-errorrootcode-for-hash-md5-was-not-found). The documented workaround to reinstall Python and openssl was not working, so I decided I would uninstall and reinstall Python.

The problem is, when you try to install Python 2 with brew, you receive this message:

```
brew install python@2
Error: No available formula with the name &quot;python@2&quot;
==&gt; Searching for a previously deleted formula (in the last month)...
Warning: homebrew/core is shallow clone. To get complete history run:
  git -C &quot;$(brew --repo homebrew/core)&quot; fetch --unshallow

python@2 was deleted from homebrew/core in commit 028f11f9e:
  python@2: delete (https://github.com/Homebrew/homebrew-core/issues/49796)
  EOL 1 January 2020.
  We gave it 1 month more to live so that people had time to migrate.
  All in all, developers had 11 years to do their migration.
  You can use the `brew extract` command and maintain python@2 in your own
  tap if necessary:
  https://docs.brew.sh/How-to-Create-and-Maintain-a-Tap

To show the formula before removal run:
  git -C &quot;$(brew --repo homebrew/core)&quot; show 028f11f9e^:Formula/python@2.rb

If you still use this formula consider creating your own tap:
  https://docs.brew.sh/How-to-Create-and-Maintain-a-Tap
```

Unfortunately I still have a number of brew formulas that depend on Brew&#39;s python@2. Those include `awscli`, `letsencrypt`, pr `sshuttle` for example

```
aws
zsh: /usr/local/bin/aws: bad interpreter: /usr/local/opt/python@2/bin/python2.7: no such file or directory
```

I don&#39;t know how to use this `brew extract` command they documented to reinstall Python@2. It needs a formula and a tap. I imagine the formula would be `python@2`. I&#39;m not sure what the tap would need to be.

Additionally reinstalling the taps such as `aws` or `letsencrypt` is not working very well either.

After reinstalling `awscli` (`brew reinstall awscli`), running aws commands still gives errors.

```
aws
/usr/local/Cellar/awscli/2.0.0/libexec/lib/python3.8/site-packages/jmespath/visitor.py:32: SyntaxWarning: &quot;is&quot; with a literal. Did you mean &quot;==&quot;?
  if x is 0 or x is 1:
/usr/local/Cellar/awscli/2.0.0/libexec/lib/python3.8/site-packages/jmespath/visitor.py:32: SyntaxWarning: &quot;is&quot; with a literal. Did you mean &quot;==&quot;?
  if x is 0 or x is 1:
/usr/local/Cellar/awscli/2.0.0/libexec/lib/python3.8/site-packages/jmespath/visitor.py:34: SyntaxWarning: &quot;is&quot; with a literal. Did you mean &quot;==&quot;?
  elif y is 0 or y is 1:
/usr/local/Cellar/awscli/2.0.0/libexec/lib/python3.8/site-packages/jmespath/visitor.py:34: SyntaxWarning: &quot;is&quot; with a literal. Did you mean &quot;==&quot;?
  elif y is 0 or y is 1:
/usr/local/Cellar/awscli/2.0.0/libexec/lib/python3.8/site-packages/jmespath/visitor.py:260: SyntaxWarning: &quot;is&quot; with a literal. Did you mean &quot;==&quot;?
  if original_result is 0:
usage: aws [options] &lt;command&gt; &lt;subcommand&gt; [&lt;subcommand&gt; ...] [parameters]
To see help text, you can run:

  aws help
  aws &lt;command&gt; help
  aws &lt;command&gt; &lt;subcommand&gt; help
aws: error: the following arguments are required: command
```

Content Type	Original Author	Original Content on Stackoverflow
Question	user200783	View Question on Stackoverflow
Solution 1 - Python	rkatkam	View Answer on Stackoverflow
Solution 2 - Python	Charlton Lane	View Answer on Stackoverflow

Python2: Should I use Pickle or cPickle?

Python Problem Overview

Python Solutions

Solution 1 - Python

Solution 2 - Python

Allow AWS Lambda to access RDS Database

iOS 9 : Warning "All interface orientations must be supported unless the app requires full screen" for universal app

Attributions