What's the purpose of tf.app.flags in TensorFlow?

I am reading some example codes in Tensorflow, I found following code

flags = tf.app.flags
FLAGS = flags.FLAGS
flags.DEFINE_float('learning_rate', 0.01, 'Initial learning rate.')
flags.DEFINE_integer('max_steps', 2000, 'Number of steps to run trainer.')
flags.DEFINE_integer('hidden1', 128, 'Number of units in hidden layer 1.')
flags.DEFINE_integer('hidden2', 32, 'Number of units in hidden layer 2.')
flags.DEFINE_integer('batch_size', 100, 'Batch size.  '
                 'Must divide evenly into the dataset sizes.')
flags.DEFINE_string('train_dir', 'data', 'Directory to put the training data.')
flags.DEFINE_boolean('fake_data', False, 'If true, uses fake data '
                 'for unit testing.')

in tensorflow/tensorflow/g3doc/tutorials/mnist/fully_connected_feed.py

But I can't find any docs about this usage of tf.app.flags.

And I found the implementation of this flags is in the tensorflow/tensorflow/python/platform/default/_flags.py

Obviously, this tf.app.flags is somehow used to configure a network, so why is it not in the API docs? Can anyone explain what is going on here?

The tf.app.flags module is presently a thin wrapper around python-gflags, so the documentation for that project is the best resource for how to use it argparse, which implements a subset of the functionality in python-gflags.

Note that this module is currently packaged as a convenience for writing demo apps, and is not technically part of the public API, so it may change in future.

We recommend that you implement your own flag parsing using argparse or whatever library you prefer.

EDIT: The tf.app.flags module is not in fact implemented using python-gflags, but it uses a similar API.

The tf.app.flags module is a functionality provided by Tensorflow to implement command line flags for your Tensorflow program. As an example, the code you came across would do the following:

flags.DEFINE_float('learning_rate', 0.01, 'Initial learning rate.')

The first parameter defines the name of the flag while the second defines the default value in case the flag is not specified while executing the file.

So if you run the following:

$ python fully_connected_feed.py --learning_rate 1.00

then the learning rate is set to 1.00 and will remain 0.01 if the flag is not specified.

As mentioned [in this article](https://medium.com/@zzzuzum/command-line-flags-python-and-tensorflow-85ab217dbd5 "Command line flags: Python and Tensorflow"), the docs are probably not present because this might be something that Google requires internally for its developers to use.

Also, as mentioned in the post, there are several advantages of using Tensorflow flags over flag functionality provided by other Python packages such as argparse especially when dealing with Tensorflow models, the most important being that you can supply Tensorflow specific information to the code such as information about which GPU to use.

Short Answer:

At Google, they use flag systems to set default values for arguments. It's similar to argparse. They use their own flag system instead of argparse or sys.argv.

Source: I worked there before.

Long Answer:

For the arguments you have in that example, they are called hyperparameters. In neural network there are multiple parameters you can optimize in order to get a a desired results. For example, for batch_size, it's the number of data vector (This can be image, text, or raw data points) that can be passed in a single shot to the optimizer.

You can Google the name of the argument, and see what's the purpose of it of it. If you want to learn about Deep Learning, I recommend you take Andrew Ng course.

When you use tf.app.run(), you can transfer the variable very conveniently between threads using tf.app.flags. See this for further usage of tf.app.flags.

After trying many times I found this to print all FLAGS key as well as actual value -

for key in tf.app.flags.FLAGS.flag_values_dict():

  print(key, FLAGS[key].value)