Can't get AWS Lambda function to log (text output) to CloudWatch

Amazon Web-ServicesAws LambdaAmazon Cloudwatch

Amazon Web-Services Problem Overview


I'm trying to set up a Lambda function that will process a file when it's uploaded to an S3 bucket. I need a way to see the output of console.log when I upload a file, but I can't figure out how to link my Lambda function to CloudWatch.

I figured about by looking at the context object that my log group is /aws/lambda/wavToMp3 and the log stream is 2016/05/23/[$LATEST]hex_code_redacted. So I created that group and stream in CloudWatch, yet nothing is being logged to it.

Amazon Web-Services Solutions


Solution 1 - Amazon Web-Services

After you update your policy, it seems that you have to update your function's settings to refresh all job instances to read new policies.

So if you just click 'test' button from Lambda console after you update your role policy in IAM, the cached Lambda instances will still have old role permissions, so you will still see no logs being written to Cloudwatch logs.

Just change your timeout by a second and click on 'save and test' button, and you will start to see logs in Cloudwatch.

Solution 2 - Amazon Web-Services

For the lambda function to create log stream and publish logs to cloudwatch, the lambda execution role needs to have the following permissions.

{
    "Statement": [
        {
            "Action": [
                "logs:CreateLogGroup",
                 "logs:CreateLogStream",
                 "logs:PutLogEvents"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:logs:*:*:*"
        }
    ]
} 

Please refer to the following AWS documentation for more details http://docs.aws.amazon.com/lambda/latest/dg/intro-permission-model.html#lambda-intro-execution-role

Solution 3 - Amazon Web-Services

> For the lambda function to create log stream and publish logs to cloudwatch, the lambda execution role needs to have the following permissions

I already had these permissions yet it did not work.

> Just change your timeout by a second and click on 'save and test' button, and you will start to see logs in Cloudwatch.

I changed the timeout, saved and logs still did not work.

I assigned another role and logs still did not work.

What ended up working for me was clicking "Create a custom role", then "Allow". This was it and logs started being generated but since I did not want to use a new role but my existing role, I simply assigned my existing role afterwards and it worked. So technically I should have returned back to original configuration that did not work but now it works. Go figure.

Solution 4 - Amazon Web-Services

Apparently another necessity for logging to happen is the Lambda function must indicate completion; for instance in the Python context, the handler must return something other than None.

Solution 5 - Amazon Web-Services

July 2020 Update !!

Logs may not be in us-east-1, try looking for lambda edge logs in different regions !!

Solution 6 - Amazon Web-Services

As other answers state you need to give lambda permission to post logs to cloud watch logs. AWS had provided AWSLambdaExecute policy just for that. It's json is -

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:*"
            ],
            "Resource": "arn:aws:logs:*:*:*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::*"
        }
    ]
}

You can add this policy in your role which is assigned to your lambda and you should start seeing the logs.

NOTE: It also has S3 read/write access. If you do not want it you can create a custom policy with just the logs part.

Solution 7 - Amazon Web-Services

Make sure you have the full path of your "Existing role" in your lambda function "Configuration":

Role: Choose an existing role Existing Role: service-role/yourRoleName

For some reason, typing only yourRoleName will work for some services (like SES) but not for CloudWatch.

Also, you may try creating a new role instead of using an existing one. This will create the role with the proper configuration (hopefully).

Solution 8 - Amazon Web-Services

For the issue was I was trying to create a log group in the Cloudformation script by : AWS::Logs::LogGroup and then trying to push the Lambda log to this log group. :P Novice After careful reading , i found that Lambda creates its own log with the aforementioned format: /aws/lambda/ We just need to provide policy permission to this log group , or just a generic permission with resource as: arn:aws:logs:::*

hope this helps

Solution 9 - Amazon Web-Services

Maybe a bit late, but for those who still struggle with seeing the lambda logs in cloudwatch. I noticed this regarding the lambda function's execution role: "You may use an existing role with this function. Note that the role must be assumable by Lambda and must have Cloudwatch Logs permissions." So in IAM i granted " CloudWatchLogsFullAccess" to the role i assigned to my function. then in cloudwatch, under logs, you'll see the logs for the functions assigned this role.

Solution 10 - Amazon Web-Services

It might already log, we just couldn't find the logs we expect...

e.g.
app.use(function simpleLogger (req, res, next) {
  console.info('[Logger]', req.method, req.originalUrl)
  next()
})

After performing GET /hello?world=1,

Local console: (simple and clear, nice!)

[Logger] GET /hello?world=1

CloudWatch Logs: (can you easily find the exact log below?)

START RequestId: a3552c34-f7a6-11e8-90ba-2fb886f31fb0 Version: $LATEST
2018-12-04T09:26:11.236Z  a3552c34-f7a6-11e8-90ba-2fb886f31fb0  [Logger] GET /hello?world=1
END RequestId: a3552c34-f7a6-11e8-90ba-2fb886f31fb0
REPORT RequestId: a3552c34-f7a6-11e8-90ba-2fb886f31fb0  Duration: 41.02 ms  Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 29 MB

Conclusion: too verbose to locate the original logs.

Solution 11 - Amazon Web-Services

I encountered this problem but none of the answers above solved my issue. It turns out that the region was somehow set to Ohio when I first started CloudWatch. After I changed it to US East (N. Virginia), everything works fine.

Solution 12 - Amazon Web-Services

CloudWatch & CloudWatch Logs are different Permissions, you need add CloudWatch Logs to the policy which attached with your role.

Solution 13 - Amazon Web-Services

There's a writeup called How to Monitor AWS Lambda with CloudWatch with a section on "How to Use CloudWatch Logs with Lambda". Looks like you already found your answer, but for anybody without the IAM specific issues, this may help.

Solution 14 - Amazon Web-Services

You might want add the Last Ingestion Time column to the log output. It's taking a few minutes for all my events to be written.

Solution 15 - Amazon Web-Services

Though this got answered already, Just wanted to add my experience which might be useful for others.

Even the permissions are set appropriately so that lambda can log in cloudwatch, sometimes it is taking 3-4 hrs to reflect the log groups.

In my case when using lambda for DynamoDB events, where lambda was given all needed permissions for both cloudwatch and DynamoDB. It took 4 hrs to get the logs reflected. May be some sync issues from AWS end. After 4 hrs without any action from my end, I was able to see the Cloudwatch logs.

Also make sure that you are seacrching the logs in the same region where your function is created.

Solution 16 - Amazon Web-Services

In my case, the permission boundary set on the role was the issue.

Although the role policy allowed logs:PutLogEvents, the permission boundary set on the role disallowed this action on the log group.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionffxsamView Question on Stackoverflow
Solution 1 - Amazon Web-ServiceshoonohView Answer on Stackoverflow
Solution 2 - Amazon Web-ServicesShibashisView Answer on Stackoverflow
Solution 3 - Amazon Web-ServicestomusiakaView Answer on Stackoverflow
Solution 4 - Amazon Web-ServicesVicView Answer on Stackoverflow
Solution 5 - Amazon Web-ServicesjellycscView Answer on Stackoverflow
Solution 6 - Amazon Web-ServicesAniket ThakurView Answer on Stackoverflow
Solution 7 - Amazon Web-ServicesLeonardo Max AlmeidaView Answer on Stackoverflow
Solution 8 - Amazon Web-ServicesAnupam MahapatraView Answer on Stackoverflow
Solution 9 - Amazon Web-ServicesClive SargeantView Answer on Stackoverflow
Solution 10 - Amazon Web-ServiceskenberkeleyView Answer on Stackoverflow
Solution 11 - Amazon Web-ServicesrandomcatView Answer on Stackoverflow
Solution 12 - Amazon Web-ServicesSherry LiView Answer on Stackoverflow
Solution 13 - Amazon Web-ServicesmbarlockerView Answer on Stackoverflow
Solution 14 - Amazon Web-ServicesC.M.View Answer on Stackoverflow
Solution 15 - Amazon Web-ServicessanthoshView Answer on Stackoverflow
Solution 16 - Amazon Web-ServicesGuyView Answer on Stackoverflow