AWS Lambda sending HTTP request

Question

This is likely a question with an easy answer, but i can't seem to figure it out.

Background: I have a python Lambda function to pick up changes in a DB, then using HTTP post the changes in json to a URL. I'm using urllib2 sort of like this:

# this runs inside a loop, in reality my error handling is much better
request = urllib2.Request(url)
request.add_header('Content-type', 'application/json')
try:
    response = urllib2.urlopen(request, json_message)
except:
    response = "Failed!"

It seems from the logs either the call to send the messages is skipped entirely, or times-out while waiting for a response.

Is there a permission setting I'm missing, the outbound rules in AWS appear to be right. [Edit] - The VPC applied to this lambda does have internet access, and the security groups applied appear to allow internet access. [/Edit]

I've tested the code locally (connected to the same data source) and it works flawlessly.

It appears the other questions related to posting from a lambda is related to node.js, and usually because the url is wrong. In this case, I'm using a requestb.in url, that i know is working as it works when running locally.

Edit:

I've setup my NAT gateway, and it should work, I've even gone as far as going to a different AWS account, re-creating the conditions, and it works fine. I can't see any Security Groups that would be blocking access anywhere. It's continuing to time-out.

Edit: Turns out i was just an idiot when i setup my default route to the NAT Gateway, out of habit i wrote 0.0.0.0/24 instead of 0.0.0.0/0

Your Lambda function is configured to run inside your VPC? Is it configured to execute inside a public subnet or a private subnet? — Matt Houser
– Matt Houser, Commented Oct 19, 2016 at 16:30
I've got 4 subnets applied, all of which have internet access. It can access other resources inside the VPC/subnet fine, as that's where the DB is. — Steve Butler
– Steve Butler, Commented Oct 19, 2016 at 16:36
Are all 4 subnets public subnets? Or are any of them private? — Matt Houser
– Matt Houser, Commented Oct 19, 2016 at 16:41
They're private, but I have setup a gateway for them to have access (I've got other EC2 machines on the subnet that access the internet fine) — Steve Butler
– Steve Butler, Commented Oct 19, 2016 at 16:45
Do you have a security group on your NAT that's possibly blocking access from your Lambda functions? — Matt Houser
– Matt Houser, Commented Oct 19, 2016 at 17:00

mfisherca · Accepted Answer · 2016-10-19 19:22:55Z

10

If you've deployed your Lambda function inside your VPC, it does not obtain a public IP address, even if it's deployed into a subnet with a route to an Internet Gateway. It only obtains a private IP address, and thus can not communicate to the public Internet by itself.

To communicate to the public Internet, Lambda functions deployed inside your VPC need to be done so in a private subnet which has a route to either a NAT Gateway or a self-managed NAT instance.

answered Oct 19, 2016 at 19:22

mfisherca

2,49923 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Steve Butler Over a year ago

I've setup a NAT Gateway, and it's still unable to access the internet. I now have it running on a private subnet that has a route to a nat gateway on a public subnet (routed though an igw).

mfisherca Over a year ago

What sort of VPC are you using--one generated from the Management Console's VPC WIzard (e.g. "VPC with Public and Private Subnets"), or a custom one? Did you set it up, or someone else? The only other part of a VPC that could be blocking the access that hasn't been mentioned yet is the network ACL (NACL). By default they're usually fully permissive, but if this is a pre-existing or custom VPC, a NACL may need to be modified as well.

Steve Butler Over a year ago

It was a stupid route issue on my part, muscle memory of typing /24 after IPs caused my default route to break, totally my bad.

Aaron Medacco Over a year ago

I did the same thing, Steve, created a bunch of private subnets with the /x and then misconfigured the route.

Putnik · Accepted Answer · 2020-05-03 10:46:09Z

0

I have also faced the same issue. I overcame it by using boto3 to invoke a lambda from another lambda.

import boto3

client = boto3.client('lambda')

response = client.invoke(
    FunctionName='string',
    InvocationType='Event'|'RequestResponse'|'DryRun',
    LogType='None'|'Tail',
    ClientContext='string',
    Payload=b'bytes'|file,
    Qualifier='string'
)

But make sure that you set the IAM policy for lambda role (in the Source AWS account) to invoke that another lambda.

Adding to the above, boto3 uses HTTP at the backend.

edited May 3, 2020 at 10:46

Putnik

7,0449 gold badges50 silver badges70 bronze badges

answered Feb 1, 2018 at 16:46

Fayaz Deen

33 bronze badges

Collectives™ on Stack Overflow

AWS Lambda sending HTTP request

2 Answers 2

4 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related