Add a schedule field to Instances to integrate with AWS Instance Scheduler [#3017858]

Add a schedule field to integrate with AWS Instance Scheduler Service described here: https://docs.aws.amazon.com/solutions/latest/instance-scheduler/welcome....

Comment	File	Size	Author
#19	3017858_5.patch	28.81 KB	baldwinlouie
#17	3017858_4.patch	29.51 KB	baldwinlouie
#15	3017858_3.patch	29.39 KB	baldwinlouie
#13	3017858_2.patch	21.44 KB	baldwinlouie
#11	schedule_name.png	329.75 KB	baldwinlouie
#11	cloud-watch-instance-scheduler.png	644.46 KB	baldwinlouie
#8	3017858_1.patch	22.11 KB	baldwinlouie
#2	3017858.patch	18.17 KB	baldwinlouie

Comments

Comment #1

4 December 2018 at 05:48

baldwinlouie created an issue. See original summary.

Comment #2

baldwinlouie commented 4 December 2018 at 18:39

Status:

Active

» Needs review

Status	File	Size
new	3017858.patch	18.17 KB

Attaching Version 1 of the patch. @yas, please try to test this on your personal environment to see if it works. I wrote a set of instructions in the README.txt portion of the patch. If you can follow along, that means it is descriptive enough for the general community. Please provide feedback if I need to make adjustments.

I think this will be tweaked a little bit before it gets checked in.

Comment #3

4 December 2018 at 18:44

Status:

Needs review

» Needs work

The last submitted patch, 2: 3017858.patch, failed testing. View results

Comment #4

yas

🇯🇵 he/him

Japanese

California 🇺🇸

commented 5 December 2018 at 15:24

@baldwinlouie

I couldn't apply the patch. Could you please take a look at it?

Checking patch modules/cloud_service_providers/aws_cloud/README.txt...
3017858.patch:57: new blank line at EOF.
+
Checking patch modules/cloud_service_providers/aws_cloud/aws_cloud.install...
Checking patch modules/cloud_service_providers/aws_cloud/aws_cloud.module...
Checking patch modules/cloud_service_providers/aws_cloud/config/install/aws_cloud.settings.yml...
Checking patch modules/cloud_service_providers/aws_cloud/config/schema/aws_cloud.settings.schema.yml...
3017858.patch:160: new blank line at EOF.
+
Checking patch modules/cloud_service_providers/aws_cloud/src/Aws/Ec2/InstanceInterface.php...
Checking patch modules/cloud_service_providers/aws_cloud/src/Entity/Ec2/Instance.php...
Hunk #1 succeeded at 554 (offset 7 lines).
Hunk #2 succeeded at 1104 (offset 7 lines).
Checking patch modules/cloud_service_providers/aws_cloud/src/Form/Config/AwsCloudAdminSettings.php...
Checking patch modules/cloud_service_providers/aws_cloud/src/Form/Ec2/InstanceEditForm.php...
Hunk #1 succeeded at 181 (offset 5 lines).
Hunk #2 succeeded at 227 (offset 5 lines).
Checking patch modules/cloud_service_providers/aws_cloud/src/Plugin/AwsCloudServerTemplatePlugin.php...
Checking patch modules/cloud_service_providers/aws_cloud/src/Service/AwsEc2Service.php...
Hunk #1 succeeded at 615 (offset 9 lines).
Hunk #2 succeeded at 628 (offset 9 lines).
Hunk #3 succeeded at 671 (offset 10 lines).
Hunk #4 succeeded at 717 (offset 10 lines).
Checking patch modules/cloud_service_providers/aws_cloud/tests/src/Functional/Ec2/InstanceTest.php...
error: while searching for:
    $this->assertNoText(t('Notice'), t('View | Make sure w/o Notice'));
    $this->assertNoText(t('warning'), t('View | Make sure w/o Warnings'));
    $this->assertText("$security_group_name1, $security_group_name2", t('Security Group'));
  }

  private function createInstanceTestData() {

error: patch failed: modules/cloud_service_providers/aws_cloud/tests/src/Functional/Ec2/InstanceTest.php:243
error: modules/cloud_service_providers/aws_cloud/tests/src/Functional/Ec2/InstanceTest.php: patch does not apply

Comment #5

baldwinlouie commented 5 December 2018 at 23:59

Status:

Needs work

» Needs review

@yas, can you please update your local instance from Git before applying? It is failing at

error: while searching for:
    $this->assertNoText(t('Notice'), t('View | Make sure w/o Notice'));
    $this->assertNoText(t('warning'), t('View | Make sure w/o Warnings'));
    $this->assertText("$security_group_name1, $security_group_name2", t('Security Group'));
  }

This code came from this issue: https://www.drupal.org/project/cloud/issues/3015916

Comment #6

yas

🇯🇵 he/him

Japanese

California 🇺🇸

commented 6 December 2018 at 15:24

@Baldwinlouie,

I still have the following error... I checked it starting from git clone...:

Cloning into 'cloud'...
remote: Counting objects: 6397, done.
remote: Compressing objects: 100% (4840/4840), done.
remote: Total 6397 (delta 4351), reused 1803 (delta 1081)
Receiving objects: 100% (6397/6397), 2.10 MiB | 550.00 KiB/s, done.
Resolving deltas: 100% (4351/4351), done.
Checking patch modules/cloud_service_providers/aws_cloud/README.txt...
3017858.patch:57: new blank line at EOF.
+
Checking patch modules/cloud_service_providers/aws_cloud/aws_cloud.install...
Checking patch modules/cloud_service_providers/aws_cloud/aws_cloud.module...
Checking patch modules/cloud_service_providers/aws_cloud/config/install/aws_cloud.settings.yml...
Checking patch modules/cloud_service_providers/aws_cloud/config/schema/aws_cloud.settings.schema.yml...
3017858.patch:160: new blank line at EOF.
+
Checking patch modules/cloud_service_providers/aws_cloud/src/Aws/Ec2/InstanceInterface.php...
Checking patch modules/cloud_service_providers/aws_cloud/src/Entity/Ec2/Instance.php...
Hunk #1 succeeded at 554 (offset 7 lines).
Hunk #2 succeeded at 1104 (offset 7 lines).
Checking patch modules/cloud_service_providers/aws_cloud/src/Form/Config/AwsCloudAdminSettings.php...
Checking patch modules/cloud_service_providers/aws_cloud/src/Form/Ec2/InstanceEditForm.php...
Hunk #1 succeeded at 183 (offset 7 lines).
Hunk #2 succeeded at 235 (offset 13 lines).
Checking patch modules/cloud_service_providers/aws_cloud/src/Plugin/AwsCloudServerTemplatePlugin.php...
Checking patch modules/cloud_service_providers/aws_cloud/src/Service/AwsEc2Service.php...
Hunk #1 succeeded at 615 (offset 9 lines).
Hunk #2 succeeded at 628 (offset 9 lines).
Hunk #3 succeeded at 671 (offset 10 lines).
Hunk #4 succeeded at 717 (offset 10 lines).
Checking patch modules/cloud_service_providers/aws_cloud/tests/src/Functional/Ec2/InstanceTest.php...
error: while searching for:
    $this->assertNoText(t('Notice'), t('View | Make sure w/o Notice'));
    $this->assertNoText(t('warning'), t('View | Make sure w/o Warnings'));
    $this->assertText("$security_group_name1, $security_group_name2", t('Security Group'));
  }

  private function createInstanceTestData() {

error: patch failed: modules/cloud_service_providers/aws_cloud/tests/src/Functional/Ec2/InstanceTest.php:243
error: modules/cloud_service_providers/aws_cloud/tests/src/Functional/Ec2/InstanceTest.php: patch does not apply
On branch 8.x-1.x
Your branch is up to date with 'origin/8.x-1.x'.

nothing to commit, working tree clean

Comment #7

yas

🇯🇵 he/him

Japanese

California 🇺🇸

commented 6 December 2018 at 15:25

Status:

Needs review

» Needs work

Comment #8

baldwinlouie commented 7 December 2018 at 00:23

Status	File	Size
new	3017858_1.patch	22.11 KB

1 file was hidden/shown/deleted

Status	File	Size
hidden	3017858.patch	18.17 KB

@yas, Sorry about that. I had to update my local environment and merge my Unit tests into InstanceTest.php. That has changed since my patch.

Comment #9

baldwinlouie commented 7 December 2018 at 03:00

Status:

Needs work

» Needs review

changing to needs review

Comment #10

yas

🇯🇵 he/him

Japanese

California 🇺🇸

commented 17 December 2018 at 00:08

@baldwinlouie

If we have the following setting,

scheduler:ec2-startstop|default
scheduler:ec2-startstop:alt|0800;none;utc;tue
scheduler:ec2-startstop:long|0800;2200;utc;mon,tue,wed,thu,fri
scheduler:ec2-startstop:late|1500;2400;utc;tue,thu,fri
scheduler:ec2-startstop:short|1400;1800;utc;sat,sun

It would be better that the dropdown list can be:

scheduler:ec2-startstop (default)
scheduler:ec2-startstop:alt (0800;none;utc;tue)
scheduler:ec2-startstop:long (0800;2200;utc;mon,tue,wed,thu,fri)
scheduler:ec2-startstop:late (1500;2400;utc;tue,thu,fri)
scheduler:ec2-startstop:short (1400;1800;utc;sat,sun)

Also the description in the setting can be:

Schedules defined in AWS Instance Scheduler. The values entered are shown in the Schedule field on instance edit form and server template launch form. Enter one value per line, in the format <strong>key|label</strong>. The key corresponds to the schedule name defined in AWS Instance Scheduler. The label is a free form descriptive value shown to users.<br />
An example configuration might be<br />
<br />
office-hours|Office Hours - Monday to Friday 9:00am - 5:00pm<br />
<br />
See Scheduler Configuration for more information.

The multiple schedules can be specified in the dropdown list?
How can we confirm whether the schedule is set or not?

Comment #11

baldwinlouie commented 10 December 2018 at 17:54

Issue summary:

View changes

Status	File	Size
new	cloud-watch-instance-scheduler.png	644.46 KB
new	schedule_name.png	329.75 KB

@yas, responding to your questions.

1. The "|" is actually a delimiter. The value on the left of the "|" is actually the "name" of the schedule in DynamoDB. The value on the right is whatever you want to display to users. If a user wants to configure it like you described, they can enter the following into the configuration

scheduler:ec2-startstop|scheduler:ec2-startstop (default)
scheduler:ec2-startstop:alt|scheduler:ec2-startstop:alt (0800;none;utc;tue)
scheduler:ec2-startstop:long|scheduler:ec2-startstop:long (0800;2200;utc;mon,tue,wed,thu,fri)
scheduler:ec2-startstop:late|scheduler:ec2-startstop:late (1500;2400;utc;tue,thu,fri)
scheduler:ec2-startstop:short|scheduler:ec2-startstop:short (1400;1800;utc;sat,sun)

The "key" _HAS_ to correspond to schedule name defined in DynamoDB. If not, the AWS Instance Scheduler will not be able to pick up the schedule correctly. See screenshot:

2. See answer for #1.

3. AWS doesn't let you attach multiple schedule tags to an instance. Therefore it is not possible to let users select multiple values from the dropdown. Instead of specifying multiple schedule tags, to configure multiple schedules, you will have to configure multiple periods and associate them to the schedule in DynamoDB. See this url for explanation https://aws.amazon.com/answers/infrastructure-management/instance-schedu... . Question #3 at the bottom.

4. To verify if this is system is working there are a few things you can look at

Log into AWS condole and verify that the "Schedule" tag is associated with the EC2 instance.
Log into AWS console, and go to CloudWatch (in the region you launched AWS Instance Scheduler) and look at the logs. See the attached screenshot.

Comment #12

yas

🇯🇵 he/him

Japanese

California 🇺🇸

commented 17 December 2018 at 00:08

@baldwinlouie,

Thank you for your explanation. I understand the "|" is actually a delimiter, but I thought it would be easy to see if the dropdown list shows the list by the more human readable way. We can have another point of view that the dropdown list shows as it is as the tag value since the IT admin knows AWS scheduler system and spec. But still it would be better to think about the same level of viewpoint for a user, not an IT admin.

As for #2, I just suggest simply cosmetically put a new line to make a paragraph or put <strong>...</strong> HTML tag.

#3, I understand that it looks an IT admin needs to configure a specific single tag value for multiple schedule.

#4, I understand it.

Thanks
Yas

Comment #13

baldwinlouie commented 12 December 2018 at 17:33

Status	File	Size
new	3017858_2.patch	21.44 KB

1 file was hidden/shown/deleted

Status	File	Size
hidden	3017858_1.patch	22.11 KB

@yas, Thank you for your suggestions. I understand your point of view. I'm attaching a patch that address #1 and #2.

Comment #14

yas

🇯🇵 he/him

Japanese

California 🇺🇸

commented 17 December 2018 at 00:08

Status:

Needs review

» Needs work

@baldwinlouie

Thank you for your understanding and the modification. Now it looks good to me.

I found I couldn't setup the schedule in ServerTemplate form. Can you add the dropdown list at ServerTemplate, too? I think it would be nice to have the one.

Comment #15

baldwinlouie commented 13 December 2018 at 21:03

Status:

Needs work

» Needs review

Status	File	Size
new	3017858_3.patch	29.39 KB

1 file was hidden/shown/deleted

Status	File	Size
hidden	3017858_2.patch	21.44 KB

@yas, Updated patch with the schedule field on the Server Template itself. The patch as an update hook to update existing instances.

Comment #16

yas

🇯🇵 he/him

Japanese

California 🇺🇸

commented 17 December 2018 at 00:08

@baldwinlouie

The patch couldn't successfully be applied. Probably I merged the patch at https://www.drupal.org/project/cloud/issues/3017858 at first? Sorry, could you make the patch again?

$ git apply -v 3017858_3.patch
Checking patch modules/cloud_service_providers/aws_cloud/README.txt...
3017858_3.patch:57: new blank line at EOF.
+
Checking patch modules/cloud_service_providers/aws_cloud/aws_cloud.install...
error: while searching for:
  $config->set('aws_cloud_notification_msg', "Your instance [aws_cloud_instance:name] has been running since [aws_cloud_instance:launch_time].  Please review if the instance still needs to be running.");
  $config->save();
}

error: patch failed: modules/cloud_service_providers/aws_cloud/aws_cloud.install:37
error: modules/cloud_service_providers/aws_cloud/aws_cloud.install: patch does not apply
Checking patch modules/cloud_service_providers/aws_cloud/aws_cloud.module...
Hunk #2 succeeded at 309 (offset 2 lines).
Hunk #3 succeeded at 419 (offset 2 lines).
Checking patch modules/cloud_service_providers/aws_cloud/config/install/aws_cloud.settings.yml...
error: while searching for:
aws_cloud_notification_criteria: 30
aws_cloud_notification_subject: '[aws_cloud_instance:name] has been running for since [aws_cloud_instance:launch_time]'
aws_cloud_notification_msg: "Your instance [aws_cloud_instance:name] has been running since [aws_cloud_instance:launch_time].  Please review if the instance still needs to be running."

error: patch failed: modules/cloud_service_providers/aws_cloud/config/install/aws_cloud.settings.yml:5
error: modules/cloud_service_providers/aws_cloud/config/install/aws_cloud.settings.yml: patch does not apply
Checking patch modules/cloud_service_providers/aws_cloud/config/install/core.entity_form_display.cloud_server_template.aws_cloud.default.yml...
Checking patch modules/cloud_service_providers/aws_cloud/config/install/core.entity_view_display.cloud_server_template.aws_cloud.default.yml...
Checking patch modules/cloud_service_providers/aws_cloud/config/install/field.field.cloud_server_template.aws_cloud.field_schedule.yml...
Checking patch modules/cloud_service_providers/aws_cloud/config/install/field.storage.cloud_server_template.field_schedule.yml...
Checking patch modules/cloud_service_providers/aws_cloud/config/schema/aws_cloud.settings.schema.yml...
error: while searching for:
      type: string
    aws_cloud_mock_data:
      type: string

error: patch failed: modules/cloud_service_providers/aws_cloud/config/schema/aws_cloud.settings.schema.yml:17
error: modules/cloud_service_providers/aws_cloud/config/schema/aws_cloud.settings.schema.yml: patch does not apply
Checking patch modules/cloud_service_providers/aws_cloud/src/Entity/Ec2/Instance.php...
Checking patch modules/cloud_service_providers/aws_cloud/src/Form/Config/AwsCloudAdminSettings.php...
error: while searching for:
      '#description' => t('Notify instance owners after an instance has been running for this period of time'),
      '#default_value' => $config->get('aws_cloud_notification_criteria')
    ];
    return parent::buildForm($form, $form_state);
  }


error: patch failed: modules/cloud_service_providers/aws_cloud/src/Form/Config/AwsCloudAdminSettings.php:91
error: modules/cloud_service_providers/aws_cloud/src/Form/Config/AwsCloudAdminSettings.php: patch does not apply
Checking patch modules/cloud_service_providers/aws_cloud/src/Form/Ec2/AwsDeleteForm.php...
Checking patch modules/cloud_service_providers/aws_cloud/src/Form/Ec2/InstanceEditForm.php...
Checking patch modules/cloud_service_providers/aws_cloud/src/Plugin/AwsCloudServerTemplatePlugin.php...
Checking patch modules/cloud_service_providers/aws_cloud/src/Service/AwsEc2Service.php...
Checking patch modules/cloud_service_providers/aws_cloud/tests/src/Functional/Ec2/InstanceTest.php...
Hunk #1 succeeded at 231 (offset 2 lines).
Hunk #2 succeeded at 325 (offset 2 lines).
Hunk #3 succeeded at 338 (offset 2 lines).
Hunk #4 succeeded at 470 (offset 34 lines).
Hunk #5 succeeded at 479 (offset 34 lines).
Hunk #6 succeeded at 539 (offset 34 lines).

Comment #17

baldwinlouie commented 14 December 2018 at 06:22

Status	File	Size
new	3017858_4.patch	29.51 KB

1 file was hidden/shown/deleted

Status	File	Size
hidden	3017858_3.patch	29.39 KB

@yas, here's a reroll with the latest code from Git.

Comment #18

yas

🇯🇵 he/him

Japanese

California 🇺🇸

commented 17 December 2018 at 00:08

Status:

Needs review

» Needs work

@baldwinlouie

Thank you for the updated patch, however unfortunately it couldn't be applied (-_-; since the other fix makes it change. Could you please re-roll again? Sorry for the inconvenience.

Comment #19

baldwinlouie commented 14 December 2018 at 16:56

Status	File	Size
new	3017858_5.patch	28.81 KB

1 file was hidden/shown/deleted

Status	File	Size
hidden	3017858_4.patch	29.51 KB

@yas, here is the re-roll.

Comment #20

baldwinlouie commented 14 December 2018 at 16:57

Status:

Needs work

» Needs review

Comment #21

yas

🇯🇵 he/him

Japanese

California 🇺🇸

commented 17 December 2018 at 00:06

Status:

Needs review

» Reviewed & tested by the community

@baldwinlouie

Thank you for the updated patch. It looks good to work! I'll merge it and mark this issue as Fixed.

Comment #22

14 December 2018 at 22:54

yas committed f140330 on 8.x-1.x authored by baldwinlouie

Issue #3017858 by baldwinlouie, yas: Add a schedule field to Instances...

Comment #23

yas

🇯🇵 he/him

Japanese

California 🇺🇸

commented 14 December 2018 at 22:54

Status:

Reviewed & tested by the community

» Fixed

Comment #24

28 December 2018 at 22:59

Status:

Fixed

» Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

Add a schedule field to Instances to integrate with AWS Instance Scheduler

Comments

Comment #1

Comment #2

Comment #3

Comment #4

Comment #5

Comment #6

Comment #7

Comment #8

Comment #9

Comment #10

Comment #11

Comment #12

Comment #13

Comment #14

Comment #15

Comment #16

Comment #17

Comment #18

Comment #19

Comment #20

Comment #21

Comment #22

Comment #23

Comment #24

News items

Our community

Documentation

Drupal code base

Governance of community