Amazon ec2 user-data, how does it work?

We are starting instances, and accessing the user-data we place. But does anybody understand the internals of this operation (from Amazon's side)? When we pass in the user-data, at what point is that data transmitted to the VM (is this a Xen feature) and where is it stored?

I first thought, it was set as the USER_DATA env var, but we can also pass it as a file. Where is that file stored? Is it generic to all instances or varies depending on AMI?

This is not a problem per se, just wanted to know how Amazon does this.

User data screenshot: http://d.pr/GZlY

The user-data is available to the instance with a simple HTTP request at this URL:

http://169.254.169.254/latest/user-data

Amazon EC2 does not put this user-data on the instance directly, though many AMIs have code that instructs the instance to download and process the user-data automatically.

See also:

• http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html

• http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html

A very easy example for everyone's understanding.

If you want to create the file /tmp/testfile.txt when the machine gets started, you can simply add these two lines on the User data field.

#!/bin/bash
touch /tmp/testfile.txt

Remember to put the #!/bin/bash at the top before your commands.

When you run the instance (Linux AMI), you can see the User data field content at /var/lib/cloud/instance/user-data.txt

Sorry to post to such an old question, but this seems like the best place to put this additional piece of information.

Most all the AWS documents describe User Data as a property in which to put instance lifecycle startup scripting, that is, the stuff you want to run only when the instance first launches.

This is usually the case, but there has been at least one other person besides myself wanting to perform different scripting on restart, say to fix a broken key or something. And guess what... you can do that using User Data.

Here is the code and the link to the AWS document...

Content-Type: multipart/mixed; boundary="//"
MIME-Version: 1.0

--//
Content-Type: text/cloud-config; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="cloud-config.txt"

#cloud-config
cloud_final_modules:
- [scripts-user, always]

--//
Content-Type: text/x-shellscript; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="userdata.txt"

#!/bin/bash
/bin/echo "Hello World." >> /tmp/sdksdfjsdlf
--//

I can find no documentation on this formatting of User Data that allows this to happen. I've tried it out and it works. I have tried to see if it runs on every startup, and it does.

So, if you think you need to do this, I recommend that you backup. Make sure you have a copy of the original User Data, and use the code provided modified to suite, and remove the code upon the next time you stop the instance (to avoid multiple runs of the script).

AWS userdata is the set of commands/data you can provide to a instance at launch time. For example if you are launching an ec2 instance and want to have docker installed on the newly launched ec2, than you can provide set of bash commands in the userdata field of aws ec2 config page.

Usecase

• Automated deployments

• Orchestrating newly launched instance

• Bootstrapping newly launched instance with chef

• AWS Autoscaling

Here is a well explained example of AWS userdata with video tutorial

Disclaimer: everything in this answer is public information.

According to their doc on user data, AWS uses cloud-init internally. You can see some interesting processes and the actual commands if you run ps aux | grep cloud-init when the user data script is still running (you can verify that by checking the output of tail -F /var/log/cloud-init-output.txt. When it's done, the last line says for me "Cloud-init v. 19.3-44.amzn2 finished at Thu, 23 Dec 2021 02:16:48 +0000. Datasource DataSourceEc2. Up 111.63 seconds").

root      3689  3.6  3.2 149792 32692 ?        Ss   02:15   0:00 /usr/bin/python /usr/bin/cloud-init modules --mode=final
root      3705  0.0  0.2  13640  2548 ?        S    02:15   0:00 /bin/sh -c (umask 0026; tee -a /var/log/cloud-init-output.log)
root      3706  0.0  0.0  13640   208 ?        S    02:15   0:00 /bin/sh -c (umask 0026; tee -a /var/log/cloud-init-output.log)
root      3707  0.0  0.0   4236   736 ?        S    02:15   0:00 tee -a /var/log/cloud-init-output.log
ec2-user  3725  0.0  0.0 119420   916 pts/0    S+   02:15   0:00 grep --color=auto cloud-init

cat /usr/bin/cloud-init shows

#!/usr/bin/python
# EASY-INSTALL-ENTRY-SCRIPT: 'cloud-init==19.3','console_scripts','cloud-init'
__requires__ = 'cloud-init==19.3'
import re
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(
        load_entry_point('cloud-init==19.3', 'console_scripts', 'cloud-init')()
    )

#!/bin/bash
yum update -y
yum install httpd -y
echo "<html><h1>webpage 1(whatever you want, give the page name here)</h1></html>"
/var/www/html/index.html
service httpd start
chkconfig httpd on