Skip to content

Latest commit

 

History

History
56 lines (37 loc) · 3.1 KB

README.md

File metadata and controls

56 lines (37 loc) · 3.1 KB

Amazon FSx for Lustre Filesystem with Security Group

Info

This recipe shows how to use CloudFormation to set up an FSx for Lustre filesystem. It also illustrates a way to configure security groups to allow access to the filesystem.

Usage

Configure a networking stack (optional)

Follow the instructions in the Large-scale HPC Networking Setup recipe. You only need to do this once per Region you want to deploy clusters and FSx filesystems in.

Create an FSx filesystem

There are two templates. One creates a "scratch" filesystem, suitable for short-term high-performance storage. The other creates a "persistent" filesystem that is high performance, but also has higher durability.

When prompted to choose a VPC and subnet, select the one where you intend to place the majority of your computing. It is possible to access FSx for Lustre filesystems across Availability Zones, but there will be higher latency and additional costs due to cross-zone traffic.

Use with AWS ParallelCluster

To connect your FSx filesystem to a ParallelCluster deployment, you will need to know its filesystem ID. You can discover this in the Outputs from your newly-created CloudFormation stack. Note that the filesystem ID and the security group are also exported, so you can easily import them into other stacks.

You can also find the filesytem ID and other details in Amazon FSx console under File systems.

# Example shared storage configuration
---
SharedStorage:
  - Name: FsxLustre0
    StorageType: FsxLustre
    MountDir: /shared/fsx
    FsxLustreSettings:
      VolumeId: fs-0123456789abcdef0

You also need to retrieve the SecurityGroupId from the CloudFormation stack output and add it to the HeadNode and Scheduling sections.

# Example of allowing head node access
---
HeadNode:
  Networking:
    AdditionalSecurityGroups:
      - sg-0123456789abcdef0

Cost Estimate

The cost to operate an FSx for Lustre filesystem will vary based on the capacity and throughput you select. For reference, a 1.2 TB, 1000 MB/s/TiB persistent filesystem will cost around $90.00 to operate for a week. The same capacity scratch filesystem will cost about $42.00.

Cleaning Up

When you are done using this resource, you can delete it by navigating to the AWS CloudFormation console and deleting the relevant stack. If you have enabled termination protection, you will need to disable it first.