Karim El Jamali
VPC Lattice with Service Network Endpoints

VPC Lattice with Service Network Endpoints

Karim El Jamali's photo
Karim El Jamali
·

6 min read

Introduction

Recently, AWS introduced some exciting enhancements to PrivateLink and VPC Lattice, which caught my attention. Since I’ve already covered VPC Lattice in a previous blog post here, this time I’ll focus specifically on the recent improvements to VPC Lattice with the introduction of Service Network Endpoints.

At a high level, Service Network Endpoints for VPC Lattice address a few existing challenges with the service. Prior to Service Endpoints, VPC Lattice services within a Service Network were only accessible within the scope of a single VPC, making it difficult to reach those services from environments outside of AWS (such as other cloud providers or on-premises infrastructure). Additionally, there was a limitation where a single VPC could only be mapped to a single Service Network, restricting the flexibility of service architecture.

Service Network Endpoints Architecture

In Figure 1, we have two AWS accounts: Account A (the consumer) and Account B (the provider). In this scenario, an EC2 instance in Account A needs to access a web server in Account B. This setup can also be used within the same AWS account to share services between multiple VPCs.

Let’s break down the key architectural components and the improvements introduced with Service Network Endpoints.

Figure1. High Level Architecture of VPC Lattice with Service Network Endpoints

Key Architectural Elements

  1. Service Network Endpoints

    One of the primary changes in the new architecture is how the Client VPC (in Account A) accesses services in the Service Network (in Account B). Instead of associating the client-side VPC directly with the Service Network, the client connects through a VPC Endpoint of type Service Network. Previously, VPC Endpoints allowed access to specific services. Now, with Service Network Endpoints, a single endpoint allows the consumer VPC to access all services within the Service Network. This simplifies the architecture, allowing a broader range of services to be consumed without the need for multiple endpoints.

  2. Resource Gateway

    The Resource Gateway is a new component introduced on the provider side (Account B). It sits between the Service Network and the internal services (like the web server), routing traffic from the Service Network to its intended destination. It is important to note that a resource gateway can have many resources behind it.

  3. Simplified Service Exposure

    In the traditional VPC Endpoint with PrivateLink setup, the provider needed to use a Load Balancer to route traffic to the service. With Service Network Endpoints, the solution relies on resource configuration which is simply a way to identify a resource via IP Address, DNS Name or even ARN bringing the ability to share RDS databases across accounts.

  4. External Access to Services

    Figure 2 shows another major advantage of this architecture. Previously, VPC Lattice relied on VPC Association which used link local addresses that were only reachable from the VPC similar to DNS resolvers. Service Network Endpoints have IP addresses from within the consumer VPC and thus are reachable from on-prem environment via Direct Connect or IPSec VPN, thus allowing clients that live outside AWS to connect to services within the service network.

Figure2. Data Center Clients accessing Services on the Service Network

Configuration Steps

The following steps outline how to configure VPC Lattice with Service Network Endpoints.

Figure3. VPC Lattice with Service Network Endpoints Configuration Steps

This is an example of a Resource Configuration for the webserver EC2 instance in the provider account (Account B), using its IP address. The configuration also specifies the TCP protocol and port ranges. Notice that the resource is linked to the Resource Gateway, as indicated by the associated Resource Gateway ID.

Figure4. Resource Configuration using IP Address with TCP Protocol & Port Ranges

Next, let's look at the consumer account (Account A), where we can see that the Service Network sn-prod has already been created. Two resources are associated with it: one for webserver and another for webserver-2.

Figure5. Resource Configuration using IP Address with TCP Protocol & Port Ranges

Finally, we create a Service Network Endpoint in the consumer account (Account A) to enable the Client VPC in the consumer account to access all services within the Service Network, which in our example include webserver and webserver-2. Note that the service Network Endpoint is mapped to the service network sn-prod.

Figure6. VPC Endpoint within Consumer Account (Account A) of type ServiceNetwork

Figure 7 shows that each resource, webserver and webserver-2, is associated with its own DNS name.

Figure7. Each of the services gets its own DNS Name

Traffic Flows

This is the architecture we will follow to go through the traffic flows from the Client EC2 instance in Account A to the webserver in Account B.

Figure8. Traffic Flow Reference Architecture

From the client EC2 instance in Account A, I started by looking up the VPC Endpoint service name for webserver and got the response 10.1.134.114.

[ec2-user@ip-10-1-9-108 ~]$ dig vpce-0ca73f93c665a3020-snra-0648a78c7ca75b972.rcfg-07a19b961bed5fbbf.4232ccc.vpc-lattice-rsc.us-east-2.on.aws
ANSWER SECTION:
vpce-0ca73f93c665a3020-snra-0648a78c7ca75b972.rcfg-07a19b961bed5fbbf.4232ccc.vpc-lattice-rsc.us-east-2.on.aws. 60 IN A 10.1.134.114

Now looking at the interface endpoints, two things caught my attention. One was the fact that endpoints have IPv4 prefixes and the second being that there are two endpoints per AZ, not sure if this is by design. I then started to look for the 10.1.134.114 and noticed that it falls within the IPv4 prefix of the last ENI ending with “dc86” and having the address “10.1.133.186”.

Figure9. VPC Endpoint with IPv4 Prefixes

I now try to Curl to the webserver via the same service name. Output is truncated.

[ec2-user@ip-10-1-9-108 ~]$ curl vpce-0ca73f93c665a3020-snra-0648a78c7ca75b972.rcfg-07a19b961bed5fbbf.4232ccc.vpc-lattice-rsc.us-east-2.on.aws 
<!DOCTYPE html> 
<html> 
<head> 
<title>Welcome to nginx!</title> 
<p><em>Thank you for using nginx.</em></p> 
</body> 
</html>

This is the Flow logs view from the client EC2 Instance as expected 10.1.9.108 (Client EC2) and 10.1.134.114 (DNS resolution of the service name) that falls within the IP Prefix range of the ENI ending with “dc86” as shown above.

2 871253670082 eni-0bb3067f09a5c76ac 10.1.9.108 10.1.134.114 51346 80 6 6 492 1737859015 1737859029 ACCEPT OK
2 871253670082 eni-0bb3067f09a5c76ac 10.1.134.114 10.1.9.108 80 51346 6 4 1069 1737859015 1737859029 ACCEPT OK

Interestingly enough, when I looked at the VPC Flow Logs from the VPC Endpoint ENI ending with dc86 perspective, I noticed now the destination address is changed to 10.1.133.186. The reason I know this is the same flow is the source port being 51346. There might be some AWS magic happening here!

2 871253670082 eni-0e4560bd9b77ddc86 10.1.9.108 10.1.133.186 51346 80 6 6 492 1737858991 1737859051 ACCEPT OK
2 871253670082 eni-0e4560bd9b77ddc86 10.1.133.186 10.1.9.108 80 51346 6 4 1069 1737858980 1737859040 ACCEPT OK

Now we look at the traffic within the provider’s account (Account B). We start by doing a tcpdump on the webserver (intended destination of the flow).

[ec2-user@ip-10-1-11-205 ~]$ sudo tcpdump port 80 
listening on ens5, link-type EN10MB (Ethernet), snapshot length 262144 bytes 
02:14:49.133013 IP ip-10-1-134-81.us-east-2.compute.internal.42379 > ip-10-1-11-205.us-east-2.compute.internal.http: Flags [S], seq 209584949, win 62727, options [mss 8460,sackOK,TS val 1329002378 ecr 0,nop,wscale 7], length 0 
02:14:49.133046 IP ip-10-1-11-205.us-east-2.compute.internal.http > ip-10-1-134-81.us-east-2.compute.internal.42379: Flags [S.], seq 1972392839, ack 209584950, win 62643, options [mss 8961,sackOK,TS val 1954915819 ecr 1329002378,nop,wscale 7], length 0

And once again, I try to find “10.1.134.81” to realize that the resource gateway also has an IPv4 prefix and the address “10.1.134.81” is within the range of the 1st ENI ending with “fece” with IP address 10.1.143.10. Thus, the resource gateway actually does a source NAT to traffic coming from the service network to its own address. Again, the question on the two ENIs per AZ remains to be answered!

Figure10. VPC Endpoint with IPv4 Prefixes

Finally, all this information can also be visible via the Resource Configuration Access Logs under Monitoring.

{
    "eventTimestamp": "2025-01-26T02:14:49.133Z",
    "serviceNetworkArn": "arn:aws:vpc-lattice:us-east-2:871253670082:servicenetwork/sn-049223690051e455d",
    "serviceNetworkResourceAssociationId": "snra-0648a78c7ca75b972",
    "vpcEndpointId": "vpce-0ca73f93c665a3020",
    "sourceVpcArn": "arn:aws:ec2:us-east-2:871253670082:vpc/vpc-028f697ee659bfbe4",
    "resourceConfigurationArn": "arn:aws:vpc-lattice:us-east-2:329599627605:resourceconfiguration/rcfg-07a19b961bed5fbbf",
    "protocol": "tcp",
    "sourceIpPort": "10.1.9.108:51676", #client VM
    "destinationIpPort": "10.1.134.114:80", #VPC Endpoint in Consumer Account
    "gatewayIpPort": "10.1.134.81:42379", #IP Address belonging to one of the ENIs of the resource gateway
    "resourceIpPort": "10.1.11.205:80" #Web Server
}

This concludes our explanation on VPC Lattice with Service Network Endpoints!

service networkingAWSvpcVPC EndpointsAzureGCPmulticloudnetworking