My source tree contains several directories which are using Git source control, and I need to tarball the whole tree excluding any references to the Git metadata or custom log files.

I thought I&#39;d have a go using a combination of find/egrep/xargs/tar, but somehow the tar file contains the .git directories and the *.log files.

This is what I have:

    find -type f . | egrep -v &#39;\.git|\.log&#39; | xargs tar rvf ~/app.tar

Can someone explain my misunderstanding here? Why is tar processing the files that find and egrep are filtering?

I&#39;m open to other techniques as well.


Tarballing without Git metadata

I would like my build script to act properly for release and development environments.

For this I would like to define a property in ant, call it (e.g.) `fileTargetName`

`fileTargetName` will get it&#39;s value from the environment variable `RELEASE_VER` if it&#39;s available, if it is not available it will get the default value of **dev**

Help with ant `&lt;condition&gt;&lt;value&gt;&lt;/condition&gt;` &amp; `&lt;property&gt;` to get it working is appreciated.

define ant property from environment with default value

Could somebody explain me the exactly differences between these two Maven plugins and when it is useful to work with one of these plugins?

What are the differences between Maven Jar Plugin and Maven Assembly Plugin?

Is there a way to use an extra Python package index (ala `pip --extra-index-url pypi.example.org mypackage`) with `setup.py` so that running `python setup.py install` can find the packages hosted on `pypi.example.org`?

Using an extra python package index url with setup.py

While this question has a python backend, the question is **not** tied to python itself, but rather about extension mechanisms and how to register/lookup for plugins.

In python, the concept of entrypoints was introduced by setuptools, and is tied to the metadata of installed python distributions (called packages in other packaging systems).

As of my understanding, one of the features provided by entrypoints is to allows an application to define a place where others can put things in, so any application wanting to use an entrypoint can get a list of registered classes/functions there. Let&#39;s take an example:

* Foo defines the entrypoint &quot;entrypoint1&quot; and look for plugins registered under this name.
* Bar register a callable (`Bar.callable`) on the &quot;entrypoint1&quot; entrypoint.
* Any python script is then able to list `Bar.callable` as one of the registered callables for &quot;entrypoint1&quot;.

With setuptools, applications registers entrypoints at install time and the information is stored in metadata related to packaging, called .egginfo (which usually contain information about the distribution name, its dependencies, and some more metadata about packaging).

I have the feeling that packaging metadata is not the right place to store this kind of information, as I don&#39;t get why this information is tied to packaging.

I&#39;m curious to hear about such entrypoints/extensions/plugins features in other languages, and especially if the concept is tied to metadata and packaging or not. And so the question is…

**Do you have examples I should look at? Could you explain why the design choices have been made that way?**

Can you see different ways to handle this problem? Do you know how this problem has already been solved in different tools? What are the downsides and the advantages of the current python implementation over others?

---

**What I&#39;ve found *so far***

I&#39;ve found in different projects, a way to create and to distribute &quot;plugins&quot;, which are especially taking care about the &quot;how do we make plugins&quot;. 

For instance, [libpeas][1] (the gobject plugin framework) defines a set of ways to extend a default behavior by specifying plugins. While this is interesting, I am just interested by the &quot;registering and finding&quot; (and eventually loading) part of it.

Here are some of my findings so far:

Libpeas defines [its own metadata file][2] (*.plugin), which stores information about the type of the callable (it is possible to have different plugins in different languages). The main information here is the name of the module to load.

Maven have [a design document][3] containing information on how stuff is managed there. Maven manages plugins with their dependencies and metadata so it seems like an interesting place to look for [how they implemented things][4].

As specified in their documentation, [maven plugins are using annotations][5] (`@goal`) on the classes, which is then used to find all the plugins registered with a particular `@goal`. While this approach is possible in static languages, it is not in interpreted languages as we only know what are all the possible classes / callables at one given point of the time, which may change.

Mercurial uses [a central configuration file][6] (`~/.hgrc`), containing a mapping of the name of the plugin to the path it can be found.

----

**Some more thoughts**

While that&#39;s not an answer to this question, it is also interesting to note how setuptools entrypoints are implemented, and how they compare in term of performance, with the mercurial ones.

When you ask for a particular entry point with setuptools, [all the metadata are read at run time and a list is built that way][7]. This means this reading can take some time if you have a lot of python distributions on your path. Mercurial on the other side *hardcode* this information into a single file, which means you have to specify the complete path to your callable there, then registered callables are not &quot;discovered&quot; but &quot;read&quot; directly from the configuration file. This allows finer grained configuration on what&#39;s should be available and what should not be and seems faster. 

On the other side, as the python path can be changed at runtime, this means that callables provided on such a way would have to be checked against the path in order to know if they should be returned or not in all situations.
 
----

**Why entrypoints are currently tied to packaging**

It is also interesting to understand why entrypoints are tied to packaging in setuptools. The main reason is that it seems useful that python distributions can register part of themselves as extending an entrypoint at installation time: then installing means also registering the entrypoints: no need for an extra registration step.

While this works fairly well in a large majority of cases (when python distributions are actually being installed) it doesn&#39;t when they are not installed or simply not packaged. In other words, from what I understand, you can&#39;t register an entrypoint at runtime, without having an .egg-info file.

  [1]: http://live.gnome.org/Libpeas
  [2]: http://git.gnome.org/browse/libpeas/tree/peas-demo/plugins/pythonhello/pythonhello.plugin
  [3]: https://github.com/apache/maven-3/blob/trunk/maven-core/plugin-manager.txt
  [4]: https://github.com/apache/maven-3/tree/trunk/maven-core/src/main/java/org/apache/maven/plugin
  [5]: http://maven.apache.org/guides/plugin/guide-java-plugin-development.html
  [6]: https://www.mercurial-scm.org/wiki/UsingExtensions
  [7]: https://bitbucket.org/tarek/distribute/src/b825c6440dc6/pkg_resources.py#cl-2583

Alternative implementations of python/setuptools entry points (extensions) in other languages/applications

Python has a confusing history of tools that can be used to package and describe projects: these include `distutils` in the Standard Library, `distribute`, `distutils2`, and `setuptools` (and maybe more). It appears that `distribute` and `distutils2` were discontinued in favor of `setuptools`, which leaves two competing standards.

To my understanding `setuptools` offers far more options (e.g. declaring dependencies, tests, etc.) than `distutils`, however it is not included in the Python standard library (yet?).

The *Python Packaging User Guide*[[1]] recommends now:

&gt; Use `setuptools` to define projects and create Source Distributions.

And explains:

&gt; Although you can use pure `distutils` for many projects, it does not support defining dependencies      on other projects and is missing several convenience utilities for automatically populating package  metadata correctly that are provided by `setuptools`. Being outside the standard library, setuptools  also offers a more consistent feature set across different versions of Python, and (unlike `distutils`), `setuptools` will be updated to produce the upcoming “Metadata 2.0” standard formats on all supported versions.
&gt;
&gt;Even for projects that do choose to use `distutils`, when pip installs such projects directly from source (rather than installing from a prebuilt wheel file), it will actually build your project using `setuptools` instead.

However, looking into various project&#39;s *setup.py* files reveals that this does not seem to be an actual standard. Many packages still use `distutils` and those that support `setuptools` often mix `setuptools` with `distutils` e.g. by doing a fallback import:

    try:
        from setuptools import setup
    except ImportError:
        from distutils.core import setup

Followed by an attempt to find a way to write a setup that can be installed by both `setuptools` and `distutils`. This often includes various ways of error-prone dependency checking, since `distutils` does not support dependencies in the setup function.

Why are people still making the extra effort to support `distutils` - is the fact that `setuptools` is not in the standard library the only reason? What are the advantages of `distutils` and are there any drawbacks of writing *setup.py* files that only support `setuptools`. 

[1]:https://packaging.python.org/guides/tool-recommendations/

setuptools vs. distutils: why is distutils still a thing?

I thought I heard that [py2exe][1] was able to do this, but I never figured it out.  Has anyone successfully done this?  Can I see your setup.py file, and what command line options you used?

Basically I&#39;m thinking of it giving me a single executable file that does something like unzips itself to maybe /temp and runs.


  [1]: http://www.py2exe.org/

py2exe - generate single executable file

I want to create a single executable from my Python project.  A user should be able to download and run it without needing Python installed.  If I were just distributing a package, I could use pip, wheel, and PyPI to build and distribute it, but this requires that the user has Python and knows how to install packages.  What can I use to build a self-contained executable from a Python project?

Create a single executable from a Python project

I am trying to write `setup.py` for my package. My package needs to specify a dependency on another Git repository.

This is what I have so far:

    from setuptools import setup, find_packages

    setup(
        name=&#39;abc&#39;,
        packages=find_packages(),
        url=&#39;https://github.abc.com/abc/myabc&#39;,
        description=&#39;This is a description for abc&#39;,
        long_description=open(&#39;README.md&#39;).read(),
        install_requires=[
            &quot;requests==2.7.0&quot;,
            &quot;SomePrivateLib&gt;=0.1.0&quot;,
            ],
        dependency_links = [
         &quot;git+git://github.abc.com/abc/SomePrivateLib.git#egg=SomePrivateLib&quot;,
        ],
        include_package_data=True,
    )

When I run:

    pip install -e https://github.abc.com/abc/myabc.git#egg=analyse

I get

&gt; Could not find a version that satisfies the requirement
&gt; SomePrivateLib&gt;=0.1.0 (from analyse) (from versions: ) No matching
&gt; distribution found for SomePrivateLib&gt;=0.1.0 (from analyse)

What am I doing wrong?





How to write setup.py to include a Git repository as a dependency

## The Situation
I’m trying to port an open-source library to Python 3.  ([SymPy][1], if anyone is wondering.) 

So, I need to run `2to3` automatically when building for Python 3. To do that, I need to use `distribute`. Therefore, I need to port the current system, which (according to the doctest) is `distutils`. 

&lt;br&gt;

## The Problem
Unfortunately, I’m not sure what’s the difference between these modules—`distutils`, `distribute`, `setuptools`. The documentation is sketchy as best, as they all seem to be a fork of one another, intended to be compatible in most circumstances (but actually, not all)…and so on, and so forth. 

&lt;br&gt;

## The Question
**Could someone explain the differences?** What am I supposed to use?  What is the most modern solution? (As an aside, I’d also appreciate some guide on porting to `Distribute`, but that’s a tad beyond the scope of the question…)


  [1]: http://sympy.org/ &quot;SymPy&quot;

Differences between distribute, distutils, setuptools and distutils2?

I was hoping to embed a Haskell interpreter using `hint` so that I could write plugins in Haskell to use with my program. I don&#39;t want to have to ship the entire Haskell platform for my executables. 

Normally, Haskell executables are pretty self-contained. For example, erasing the `PATH` does not cause a problem:

    $ PATH=. Hello
    Hello world

However, a simple test program using `runInterpreter` bombs if I erase the `PATH`:

    $ PATH=. TryHint
    GhcException &quot;panic! (the &#39;impossible&#39; happened)\n  (GHC version 7.8.3 for x86_64-apple-darwin):\n\tDynamic linker not initialised\n\nPlease report this as a GHC bug:  http://www.haskell.org/ghc/reportabug\n&quot;

**What libraries or executables have to be available in the environment for it to work?**

`otool` doesn&#39;t give much guidance:

    otool -L TryHint
    TryHint:
    	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)
    	/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
    	/usr/local/lib/libgmp.10.dylib (compatibility version 13.0.0, current version 13.0.0)

The test code for `TryHint` does not do much:

    import Control.Monad
    import Language.Haskell.Interpreter

    main = do
      f &lt;- runInterpreter $ loadModules [&quot;Test&quot;] &gt;&gt; setTopLevelModules [&quot;Test&quot;] &gt;&gt; interpret &quot;f&quot; (as :: Int -&gt; Int)
      case f of
        Left e -&gt; print e
        Right r -&gt; mapM_ (print . r) [1..10]

It just binds `f` to a function in `Test.hs` to be interpreted at run-time. `Test.hs` looks like this:

    module Test where
    f :: Int -&gt; Int
    f x = x + 1


How to ship an executable with Language.Haskell.Interpreter?

### Background
I was about to try Python package downloaded from GitHub, and realized that it did not have a `setup.py`, so I could not install it with
```
pip install -e &lt;folder&gt;
```
Instead, the package had a **`pyproject.toml`** file which seems to have very similar entries as the `setup.py` usually has.

### What I found
Googling lead me into [PEP-518] and it gives some critique to `setup.py` in [Rationale] section. However, it does not clearly tell that usage of `setup.py` should be avoided, or that `pyproject.toml` would as such completely replace `setup.py`.

### Questions
Is the `pyproject.toml` something that is used to replace `setup.py`? Or should a package come with both, a `pyproject.toml` and a `setup.py`?  
How would one install a project with `pyproject.toml` in an editable state?

[PEP-518]: https://www.python.org/dev/peps/pep-0518/
[Rationale]: https://www.python.org/dev/peps/pep-0518/#rationale

What is pyproject.toml file for?

I&#39;d like to make a Python package containing some [Cython][1] code. I&#39;ve got the the Cython code working nicely. However, now I want to know how best to package it.

For most people who just want to install the package, I&#39;d like to include the `.c` file that Cython creates, and arrange for `setup.py` to compile that to produce the module. Then the user doesn&#39;t need Cython installed in order to install the package.

But for people who may want to modify the package, I&#39;d also like to provide the Cython `.pyx` files, and somehow also allow for `setup.py` to build them using Cython (so those users _would_ need Cython installed).

How should I structure the files in the package to cater for both these scenarios?

The [Cython documentation gives a little guidance][2]. But it doesn&#39;t say how to make a single `setup.py` that handles both the with/without Cython cases.


  [1]: http://cython.org/
  [2]: http://docs.cython.org/src/userguide/source_files_and_compilation.html#distributing-cython-modules

How should I structure a Python package that contains Cython code

I have the source of a program (taken from cvs/svn/git/...) and I&#39;d like to build a Debian/Ubuntu package for it. The package is present in the repositories, but:

 * It is an older version (lacking features I need)
 * I need slightly different compile options than the default.

What is the easiest way of doing it? I am concerned about a couple of things
 
 * How can I check if I have listed all the dependencies correctly? (I can get some hints by looking on what the older version depended, but new dependencies may have been added.)
 * How I can I prevent the update system installing the older version in the repo on an update?
 * How I can prevent the system installing a newer version (when its out), overwriting my custom package?

How to build a Debian/Ubuntu package from source?

I have questions about egg files in Python.

I have much Python code organized by package and I&#39;m trying to create egg files.
I&#39;m following [instructions](http://peak.telecommunity.com/DevCenter/PythonEggs#building-eggs), but they are very common.

According to that, it seems I need to have a setup.py file.

1. Would you please tell me what I need to put into setup.py file and where it should reside?
2. I suppose it&#39;s enough to create setup.py and then start &quot;setup.py bdist_egg&quot; for getting egg file. Could you please confirm?
3. Is it possible to include only .pyc files into egg file?
4. Having .egg file how I can just start the code from it without unpacking like `java -jar &lt;jar file&gt;` does?


How to create Python egg file

There is a ton of information on how to do this, but since *&quot;there is more than one way to skin a cat&quot;*, and all the tutorials/manuals that cover a bit of the process seem to make certain assumptions which are different from other tutorials, I still didn&#39;t manage to grasp it.

So far this is what I think I understood.

1. My final goal should be that of creating a &quot;binary&quot; .deb package. Such package will be platform-independent (32/64 bit) as all Python programs are such.
2. To create a &quot;binary&quot; package I need first to create a source package.
3. To create the source package I can use either `CDBS` or `debhelper`. Debhelper is the recommended way for beginners.
4. The core of creating a source package is populating the `DEBIAN` directory in the source directory with a number of files clarifying where files need to be copied, what copyright and licensing scheme they are subject to, what dependencies they have, etc...
5. Step #4 can be largely automated the `dh_make`command *if* the Python source also comes with a distutils&#39; `setup.py` script.

Now my questions:

1. **Is my understanding of the process correct?** Is there anything I am missing, or anything that I got wrong?
2. Step #5 is really the more confusing to me: specifically the two points that remains most obscure to me are:
   - **How do I write a `setup.py` script that install a stand-alone programme?** **EDIT:** By _standalone programme_ I mean a program intended to be used by a desktop user (as opposed to a _module_ which I understand like a collection of functionality to be used by other software after having been imported). In my specific case I would actually need *two* such &quot;programs&quot;: the main software and a separate utility (in effect a second &quot;program&quot; that should be in the same package with the other one).
   - **What are the specificities of such a script for DEB packages?** The official documentation only seems to deal with RPM and Windows stuff...

BTW: These are the best sources of information that I could find myself so far. If you have anything better than this, please share! :)

- [Ubuntu&#39;s Python packaging guide][1]
- [Creating a .deb package from a python setup.py][2] (it _shows_ the steps, but it doesn&#39;t _explain_ them enough for me to follow along)
- [ShowMeDo video on &quot;creating a .deb package out of a python
  program&quot;][3] (it doesn&#39;t seem up-to-date and - if I got it right - will produce packages for personal use, without dependencies
and without a signed changelog and other key data that will make it incompatible with the Debian policy).

  [1]: https://wiki.ubuntu.com/PackagingGuide/Python#The_debhelper_way
  [2]: http://ghantoos.org/2008/10/19/creating-a-deb-package-from-a-python-setuppy/
  [3]: http://showmedo.com/videotutorials/series?name=linuxJensMakingDebSeries


Is there a standard way to create Debian packages for distributing Python programs?

I am new to Node.js programming and I have recently created a sample working web application using (express, backbone &amp; other complimentary view technologies, with mongoDB). Now i am at a point where I want to deploy the same on a staging environment and I am not sure how to package this application and distribute the same. [I can take care of mongoDb and setting it up seperately]

I am from Java world and in there we create jars for reusable libs and war/ear packages for web applications which is deployed in a servlet container. Now in this case since node.js itself acts as a web container as well, how do i package my webapp? 

1. Is there any standard format/guidelines of packaging node webapps built using express? (Is there a similar jar/war packaging systems for node apps?)
2. How do I deploy it once packaged? Would it become an exe, since it is also its own container? 

PS: As of now I am thinking of just manually copying all the required source files into the staging environment and run npm commands to download all dependencies on that machine and then use &#39;forever&#39; or some other mechanism to run my server.js. (Also, add some sort of monitoring, just in case app crashes and forever fails) I am not sure if that is the right way? I am sure there must be some standardized way of addressing this problem.

How to package &amp; deploy Node.js + express web application?

Is there a way to force maven(2.0.9) to include all the dependencies in a single jar file?

I have a project the builds into a single jar file. I want the classes from dependencies to be copied into the jar as well.

**Update:** I know that I cant just include a jar file in a jar file. I&#39;m searching for a way to unpack the jars that are specified as dependencies, and package the class files into my jar.

Including dependencies in a jar with Maven

 I wrote a script that will help a Windows user in her daily life. I want to simply send her the .exe and not ask her to install python, dlls or have to deal with any additional files.

I&#39;ve read plenty of the stackoverflow entries regarding compiling Python scripts into executable files. I am a bit confused as there are many options but some seem dated (no updates since 2008) and none were simple enough for me not to be asking this right now after a few hours spent on this. 

I&#39;m hoping there&#39;s a better, up-to-date way to do this.

I looked into:

- [pylunch](http://code.google.com/p/pylunch/)
- [py2exe](http://www.py2exe.org/)
- [cx_Freeze](http://cx-freeze.sourceforge.net/)
- [py2app](http://svn.pythonmac.org/py2app/py2app/trunk/doc/index.html) (only for Mac)
- [pyinstaller](http://www.pyinstaller.org/)
- [bbfreeze](http://pypi.python.org/pypi/bbfreeze/)

but either I couldn&#39;t get them to work or couldn&#39;t understand how to get the result I need. The closest I got was with py2exe but it still gave me the MSVCR71.dll

I would appreciate a step-by-step answer as I was also unable to follow some of the tweaking answers here that require some prior understanding of how to use py2exe or some of the other tools.

I&#39;m using Python 2.5 as one of the modules is only available for that version.

Process to convert simple Python script into Windows executable

I bought a third-party Java library which includes a JAR file and two DLL files. I wrote my own Java program which invoke the third-party JAR file. Now my question is **how can I package all my code into a single JAR file which include all my code and the third-party JAR and DLLs?**

I know *SWT* is such a case. The `swt.jar` includes dll files, but I don&#39;t know how to do this and how to make it work properly.

How to make a JAR file that includes DLL files?

The setuptools documentation only states:

&gt; For maximum performance, Python packages are best installed as zip files. Not all packages, however, are capable of running in compressed form, because they may expect to be able to access either source code or data files as normal operating system files. So, setuptools can install your project as a zipfile or a directory, and its default choice is determined by the project&#39;s zip_safe flag ([reference][1]).

In practical terms, what is the performance benefit gained? Is it worth investigating if my projects are zip-safe, or are the benefits generally minimal?


  [1]: http://peak.telecommunity.com/DevCenter/setuptools#setting-the-zip-safe-flag &quot;Setting the zip-safe flag&quot;

What is the advantage of setting zip_safe to True when packaging a Python project?

I am reading Maven documentation and came across the name `uber-jar`. 

What does an `uber-jar` mean and what are its features/advantages? 

What is an uber jar?

I need to package my Python application, its dependencies and Python into a single MSI installer. The end result should desirably be:

 - Python is installed in the standard location
 - the package and its dependencies are installed in a separate directory (possibly site-packages)
 - the installation directory should contain the Python uncompressed and a standalone executable is not required

All Packaging Solutions on Gang of Coders

Total of 23 Packaging Solutions