Amazon S3 Non-public Access for MagicAI Storage
We need a way to interact with S3, using non-public access. As I understand, to interact with S3 securely without needing to make the objects public, we have 3 options: Leverage pre-signed URLs Temporary security credentials, or Amazon CloudFront with S3 as an origin. Is this possible with MagicAI? If possible, this might work, using Laravel's built-in Amazon S3 integration. 1. Overview: Objective: To access S3 objects securely without public access by utilizing Laravel's Amazon S3 Integration. Tools: Amazon S3: Storage service to store objects. Amazon IAM: To create roles and users and assign permissions. Laravel Framework: The application framework in use. AWS SDK: To interact with AWS Services programmatically. 2. AWS Side Configuration: (in my AWS account) a. IAM Role/User: Create an IAM role/user with the necessary permissions. Assign this role to the EC2 instance (or in my case, for now Amazon Lightsail) where the Laravel app is hosted, or use access and secret keys to authenticate. in .json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:GetObject", "s3:PutObject", "s3:DeleteObject" ], "Resource": "arn:aws:s3:::your-bucket-name/*" } ] } 3. Laravel Side Configuration: (MagicAI app) a. Environment File (.env): Set up AWS credentials and default S3 bucket in the Laravel .env file. in .env AWSACCESSKEYID=YOURACCESSKEYID AWSSECRETACCESSKEY=YOURSECRETACCESSKEY AWSDEFAULTREGION=YOURDEFAULTREGION AWSBUCKET=YOURBUCKET_NAME b. Filesystem Configuration (config/filesystems.php): Ensure the s3 disk is configured correctly in the filesystems.php configuration file. in php 's3' => [ 'driver' => 's3', 'key' => env('AWSACCESSKEY_ID'), 'secret' => env('AWSSECRETACCESS_KEY'), 'region' => env('AWSDEFAULTREGION'), 'bucket' => env('AWS_BUCKET'), ], 4. Laravel Implementation Example: a. Generating Pre-Signed URLs: Use Laravel’s built-in Storage facade to create a pre-signed URL for temporary access to the objects. in .php use Illuminate\Support\Facades\Storage; $url = Storage::disk('s3')->temporaryUrl( 'file1.jpg', now()->addMinutes(5) ); b. Using Temporary Security Credentials: I can use an IAM role, that I already created (but the instructions say create an IAM user; a role is better) from the prior instructions. Using an IAM role with an EC2 instance, the AWS SDK can automatically use the role’s temporary security credentials. in .php // Fetching an Object from the S3 bucket. $s3 = Storage::disk('s3'); $file = $s3->get('file1.jpg'); c. Using AWS SDK for Custom Solutions: I see that there is an SDK directory in the MagicAI code... so I'm assuming that you can directly use AWS SDK for PHP for any custom solutions or more advanced scenarios. 5. Recommendation: No hard coded AWS Access Keys within the code. Use IAM Roles or Environment variables. Leverage AWS SDK to interact with AWS Services. Consider using Amazon CloudFront if there is a need to distribute content with low latency and high data transfer speeds. I'm thinking, by using the appropriate IAM roles and leveraging Laravel’s built-in functionalities, you can ensure secure access to S3 objects without needing to make them public, thereby adhering to best practices and security policies.
