Support sensu check format: https://sensuapp.org/docs/latest/reference/checks.html#example-check-res...

https://sensuapp.org/docs/latest/reference/checks.html#check-result-chec...

The basic idea is allowing to pipe the output to the sensu client: https://sensuapp.org/docs/0.25/reference/clients.html#client-socket-input

Note that while --output=json prints out a single JSON blob, this should print out a single sensor/check per line, as it will need to be separated to be passed to sensu.

Keys that should be printed:
* name (sensor config name)
* status
* ttl (optional, based on --sensu-ttl if provided)
* output
* interval (caching time, if given)
* executed (current time)
* duration: execution time
* source (optional, based on --sensu-source if given)

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Berdir created an issue. See original summary.

Ginovski’s picture

Assigned: Unassigned » Ginovski
Issue summary: View changes
Status: Active » Needs review
FileSize
3.15 KB

I am not sure about any of this:
1. Added the drush command 'monitoring-list' and added options for (output, sensu-ttl, sensu-source)
2. Added method: monitoring_drush_list(), and separate methods for table and sensu output.
To do: send results 1 by 1 in foreach to check result input via the Sensu client socket.
Not sure how to do this, any hints?

miro_dietiker’s picture

Status: Needs review » Needs work

The command already exists, it's monitoring-sensor-config.
You just need to add an option that switches the output format!

Berdir’s picture

You also don't have to worry about feeding it to sensu. I'll take care of that, just make sure that the output is correct.

Ginovski’s picture

Status: Needs work » Needs review
FileSize
5.32 KB

1. Fixed the command, added the options to the existing monitoring-sensor-config drush command.
2. Added options for output, sensu-ttl, sensu-source.
3. Added check for the output in the monitoring_drush_sensor_config method.
4. Added the 2 methods for sensu output, (method for all sensors, method for specific sensor).
Examples to use the command with the options:
- drush monitoring-sensor-config user_new --output=sensu --sensu-ttl=10 --sensu-source=admin
- drush monitoring-sensor-config --output=sensu
5. In the functions for sensu, I print out the sensu input formatted as in the example client socket usage shown here:
https://sensuapp.org/docs/0.25/reference/clients.html#client-socket-input
It's still left to pipe the sensu formatted input to the sensu client:

Berdir’s picture

Status: Needs review » Needs work
+++ b/monitoring.drush.inc
@@ -112,12 +117,33 @@ function monitoring_drush_command() {
 function monitoring_drush_sensor_config($sensor_name = NULL) {
-  if (empty($sensor_name)) {
-    monitoring_drush_sensor_config_all();
+  $output = drush_get_option('output', 'table');
+
+  // Setting sensu options, if provided.

this is still the wrong place, didn't check miros feedback :)

This is about sensor *config*. We want the results, not the configuration.

monitoring_drush_run() is the function you need to extend, which as I said, already has the output option, we just need another elseif there.

miro_dietiker’s picture

Yeah sorry. I knew the command existed, but wrongly mentioned the sensor config list instead of the sensor run.

But i also think that this issue title / summary doesn't properly mention the scope. :-)

Berdir’s picture

Title: Add --output=sensu to monitoring-list » Add --output=sensu to monitoring-run

Better now? ;)

I thought I'd mentioned that this issue isn't about actually passing the data to sensu, forgot about that. We might want to mention https://sensuapp.org/docs/latest/reference/clients.html#example-client-s... in the description for it, though.

Berdir’s picture

Title: Add --output=sensu to monitoring-run » Add --output=sensu to drush monitoring-run
Ginovski’s picture

Status: Needs work » Needs review
FileSize
3.2 KB

1. Added the options in monitoring-run command.
2. Added a function for output in sensu format.
Right now the function's output is in table format, but I have this code for the example in the upper link in #5 and #8 comment socket usage:

 /* ----- Sensu format ----- */
    $sensu_output = '{"name": "' . $name . '", "status": "' . $result->getStatus();
    if ($ttl) {
      $sensu_output .= '", "ttl": "' . $ttl;
    }
    $sensu_output .= '", "output": "' . $result->getVerboseOutput();
    $sensu_output .= '", "interval": "' . \Drupal::service('date.formatter')->formatInterval($result->getSensorConfig()->getCachingTime());
    $sensu_output .= '", "executed": "' . date('d.m.Y H:i:s',$result->getTimestamp());
    $sensu_output .= '", "duration": "' . $result->getExecutionTime();
    if ($source) {
      $sensu_output .= '", "source": "' .$source . '"}';
    }
    else {
      $sensu_output .= '"}';
    }
    drush_print($sensu_output);

Which output should I use in the function?

Berdir’s picture

Status: Needs review » Needs work

As discussed.

Ginovski’s picture

Status: Needs work » Needs review
FileSize
3.32 KB
4.66 KB

1. Fixed the examples in the drush command. (from monitoring-result to monitoring-run).
2. Added example for the sensu output.
3. Added the correct output in the function.
4. I see use of unknown function getSensorName from the SensorResultInterface in the json output function and it doesn't seem to be working. Should I fix that here or create a followup?

miro_dietiker’s picture

Status: Needs review » Needs work
+++ b/monitoring.drush.inc
@@ -190,6 +194,8 @@ function monitoring_drush_run($sensor_name = NULL) {
+  $source = drush_get_option('sensu-source');
+  $ttl = drush_get_option('sensu-ttl');

@@ -213,6 +219,9 @@ function monitoring_drush_run($sensor_name = NULL) {
+  elseif ($output == 'sensu') {

I think you should get the sensu specific options below in the sensu specific part.

Berdir’s picture

  1. +++ b/monitoring.drush.inc
    @@ -62,14 +63,17 @@ function monitoring_drush_command() {
    +      'drush monitoring-run --output=sensu --sensu-source=admin --sensu-ttl=10' => 'Will output the sensor results in sensu format. The option --sensu-source=admin will provide the sensu source. The option --sensu-ttl=10 will provide the sensor time to live.'
    

    those are not really good examples. I'd use example.org as source, and ttl 600 oder so. 10 would be just 10 seconds.

  2. +++ b/monitoring.drush.inc
    @@ -236,6 +245,38 @@ function monitoring_drush_run($sensor_name = NULL) {
    +    $sensu_output['status'] = $result->getStatus();
    

    status is 0,1 or 2. See https://sensuapp.org/docs/0.25/reference/checks.html#sensu-check-specifi... (use 3 for unknown)

  3. +++ b/monitoring.drush.inc
    @@ -236,6 +245,38 @@ function monitoring_drush_run($sensor_name = NULL) {
    +    $sensu_output['output'] = $result->getVerboseOutput();
    

    not verbose output, getMessage()

  4. +++ b/monitoring.drush.inc
    @@ -236,6 +245,38 @@ function monitoring_drush_run($sensor_name = NULL) {
    +    $sensu_output['interval'] = \Drupal::service('date.formatter')->formatInterval($result->getSensorConfig()->getCachingTime());
    

    formatInterval() is for humans. sensu is not a human, it wants seconds :)

  5. +++ b/monitoring.drush.inc
    @@ -236,6 +245,38 @@ function monitoring_drush_run($sensor_name = NULL) {
    +    $sensu_output['executed'] = date('d.m.Y H:i:s',$result->getTimestamp());
    

    same, timestamp.

We also need something else, --sensu-handlers=foo,bar, that is then added as a nested array: https://sensuapp.org/docs/0.25/reference/checks.html#check-attributes

And we also need metrics. this means, for each sensor that is numeric, add a second check result that has name $name_metric. and 'type' => 'metric'. See http://www.programblings.com/2013/11/14/sensu-checks-to-report-metrics/. The ouptut that you provide should then be in the format "$reversed_source.$name $value $executed" $reversed_source is implode('.', array_reverse(explode('.', $source)), meaning, foo.example.org becomes org.example.foo.

Ginovski’s picture

Status: Needs work » Needs review
FileSize
6.21 KB
6.21 KB

1. Fixed 1-5 and added the sensu options in the output=sensu only.
2. Added options:
- sensu-handlers
- sensu-metric-handlers
- sensu-metrics
3. Added host name as default sensu source.
4. Added second metric check result for the numeric sensors.

Ginovski’s picture

1. Removed executed from the output.
2. Changed ttl from string to integer.

Berdir’s picture

Status: Needs review » Needs work
  1. index 0000000..1712856
    --- /dev/null
    
    --- /dev/null
    +++ b/add_output_sensu_to-2788329-12.patch
    
    +++ b/add_output_sensu_to-2788329-12.patch
    @@ -0,0 +1,93 @@
    
    @@ -0,0 +1,93 @@
    +diff --git a/monitoring.drush.inc b/monitoring.drush.inc
    +index 380db17..79f22bc 100644
    +--- a/monitoring.drush.inc
    ++++ b/monitoring.drush.inc
    +@@ -7,6 +7,7 @@
    + use Drupal\Component\Serialization\Json;
    

    https://imgflip.com/i/1a6asd

  2. +++ b/monitoring.drush.inc
    @@ -236,6 +257,64 @@ function monitoring_drush_run($sensor_name = NULL) {
    +  $status_codes = [
    +    'OK' => 0,
    +    'WARNING' => 1,
    +    'CRITICAL' => 2,
    +    'UNKNOWN' => 3,
    +    'INFO' => 3,
    +  ];
    

    should use the constants. also INFO should be mapped to OK I think.

Ginovski’s picture

Status: Needs work » Needs review
FileSize
2.27 KB
6.3 KB

Didn't see that with the patches, my bad :)
1. Changes from #16 again and added the status_codes as constants.
2. Changed INFO to be mapped to OK.
3. Did additional fix for the sensu-metrics.

Ginovski’s picture

Changed the metric name in the output.

  • Berdir committed 7de6589 on 8.x-1.x authored by Ginovski
    Issue #2788329 by Ginovski, Berdir: Add --output=sensu to drush...
Berdir’s picture

Status: Needs review » Fixed
Berdir’s picture

Version: 8.x-1.x-dev » 7.x-1.x-dev
Status: Fixed » Patch (to be ported)

  • Berdir committed 6ba8d14 on 7.x-1.x authored by Ginovski
    Issue #2788329 by Ginovski, Berdir: Add --output=sensu to drush...
Berdir’s picture

Status: Patch (to be ported) » Fixed

Ported to 7.x

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.