Pushkin AWS init failed: AWS S3 PutBucketPolicy access denied

[ellissc]

Hello, I was pushing some new updates to an experiment I've been hosting on aws using pushkin. The pushkin aws init command failed multiple times, even when the --force tag is used. The error (attached) mentions that the PutBucketPolicy has an access denied error. I've ensured that everything is deleted after the armageddon command, which doesn't fix the error.

I was able to run pushkin aws init without issue around two weeks ago. It seems that there was an update on AWS' side regarding S3 bucket security policy recently (https://aws.amazon.com/about-aws/whats-new/2023/04/amazon-s3-security-best-practices-buckets-default/), has anyone else ran into this issue?

error_log.txt

[jkhartshorne]

@ellissc Sounds like it's an AWS issue that we'll need to hunt down. (Do let us know if you figure it out yourself in the meantime!). It's good timing with the semester ending; we'll be able to spend more time on this sort of thing starting next week (just to give you a sense of timeline).

In the meantime, can you confirm that you have not updated the CLI? We do know there are some bugs that crept into the CLI during the last update, which we've been trying to track down. For instance, see here.

[ellissc]

Thank you for the response!

Based on pushkin --version, we're running version 2.2.10, but don't have the pushkin prep hang issue. I haven't updated it in the past year or so that I've been using it.

In the meantime, where is the code for the pushkin aws init command? I searched through the repo and couldn't find it, but I may have missed something. I did some more looking on the AWS updates, and it seems that the changes to S3 buckets were regarding the public access block and disabling of ACLs (https://aws.amazon.com/blogs/aws/heads-up-amazon-s3-security-changes-are-coming-in-april-of-2023/), which they say can be fixed with calls like DeletePublicAccessBlock and DeleteBucketOwnershipControls after bucket creation.

[jkhartshorne]

Meantime, where the code is. Look at the repo for pushkin-cli. The offending code is probably in src/commands/aws/index.js. But I should explain how the code gets called.

We used Commander.js to build the CLI. In the repo, go to src/index.js. That lays out the basic structure of the CLI. Find the following:

`async function main() {
// program
// .option('-d, --debug', 'output extra debugging')
// .option('-s, --small', 'small pizza size')
// .option('-p, --pizza-type ', 'flavour of pizza');

program
.command('install ')
.description(Install template website ('site') or experiment ('experiment').)
.action((what) => {
if (what == 'site' | what == 'experiment'){
try {
handleInstall(what)
} catch(e) {
console.error(e)
process.exit()
}
}else{
console.error(Command not recognized. Run 'pushkin --help' for help.)
}
});

program
.command('aws ')
.description(For working with AWS. Commands include:\n init: initialize an AWS deployment.\n update: update an AWS deployment.\n armageddon: delete AWS resources created by Pushkin.\n list: list AWS resources created by Pushkin (and possibly others).)
.option('-f, --force', 'Applies only to init. Resets installation options. Does not delete AWS resources (for that, use aws armageddon).', false)
.action(async (cmd, options) => {
moveToProjectRoot();
switch (cmd){
case 'init':
try {
setEnv(false) //asynchronous
await handleAWSInit(options.force);
} catch(e) {
console.error(e)
process.exit();
}
break;
case 'update':
try {
await handleAWSUpdate();
} catch(e) {
console.error(e);
process.exit();
}
break;
case 'armageddon':
try {
await handleAWSArmageddon();
} catch(e) {
console.error(e);
process.exit();
}
break;
case 'list':
try {
await handleAWSList();
} catch(e) {
console.error(e);
process.exit();
}
break;
default:
console.error("Command not recognized. For help, run 'pushkin help aws'.")
}
});
`

This may be intuitive just from looking at it. Basically, it's a list of objects called 'programs'. The .command function specifies the CLi keyword. So the first one listed above is the code for pushkin install. The second is the one you want, pushkin aws.

In each case, there are additional arguments. You see below that there is a handler for the arguments. In the case of pushkin aws, it's the switch (cmd) section. You can see that possible arguments are init, update, armageddon, and list. Each one then calls a function. In the case of init, it calls handleAWSInit. The code for that is elsewhere in index.js.

If you follow through that code, you'll see that it eventually calls awsInit. That function is imported from /commands/aws/index.js.

I'm not sure if the actual issue is going to be in the main index.js or the commands/aws/index.js code.

One thing to know is that the current version of the repo is (as I mentioned elsewhere) broken. I recommend pulling the version that you are using (2.2.10). Make a branch of that. And then try to fix that. You'll want to build the package using yarn before trying to run anything. Basically, you build the package and then instead of using the pushkin aws init alias, you call your build of the CLI directly. I don't remember immediately how to do that. I think there's some executable that you call directly instead of just typing pushkin, but I'd need to remember / look back at the yarn documentation.

There's some information in the pushkin docs, but not much.

[ellissc]

Thank you for the info! I'll try my hand at it.

[jkhartshorne]

@ellissc any luck?

We've been in the process of rebuilding the CLI. A number of things stopped working, and it turns out that migrating to newer versions of packages involves a lot of breaking changes. If you monitor "Issues", you'll see a lot of activity right now.

[ellissc]

I haven't had a chance yet, I've been focusing on learning Node and React to get a better understanding of the communication between Pushkin and AWS.

[jkhartshorne]

So this turns out to be a fairly large change to the code, and there aren't any good tutorials. I started working on it yesterday (finally finished fixing a bunch of other bugs), and will be working on it most of today.

[jkhartshorne]

@ellissc so I think we may have fixed this. I need to do some more testing to be sure. See #197

[ellissc]

Okay, so we would need to use that branch mentioned in #197 until it gets integrated into the master branch?

[jkhartshorne]

It's integrated!

[ellissc]

Great, thank you!