librenms/poller.php

179 lines
5.8 KiB
PHP
Raw Normal View History

#!/usr/bin/env php
<?php
/**
* poller.php
*
* -Description-
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*
* @copyright (C) 2006 - 2012 Adam Armstrong
*
* Modified 4/17/19
* @author Heath Barnhart <hbarnhart@kanren.net>
*/
2021-11-18 01:23:55 +00:00
use App\Action;
use App\Actions\Device\UpdateDeviceGroupsAction;
use LibreNMS\Alert\AlertRules;
use LibreNMS\Config;
Refactor Datastores to allow future improvements. OpenTSDB Tags. (#11283) * Datastores to object oriented code, using the Laravel IoC container Change instantiation better DI move OpenTSDB Small re-orgs remove unused stuff Fix graphs and other scripts Use DI for all except rrd fix up connection error handling Add tests, fix up a "few" things Add Config::forget() Style fixes Don't reference legacy code remove accidental code paste Add datastores phpunit groups some tests * rebase fixes * some test fixes * shorter tests * shorter tests * Don't except when rrdtool can't be started. * restore tests * fix rrd tests * fix iterable change upstream * fix isValidDataset * fix invalid data bug * fix mysql incorrect ds * fix issue with data that is too long * use regular data_update() * Use log facade * OpenTSDB mis-ordered arguments fix * Making a singleton with different options makes different singletons. Just use the global config settings to disable datastores. * only filter tags for datastores that won't it don't modify the tags permanently * Update copyrights to include original authors. * Stats for all datastores * Fix mysql sends different rrd / other ds names * fix snmp last stats not initialized remove unused function * remove unused function and move single use function closer to its use * InfluxDB does not need to update null or U values. Skip write if all fields are empty * Fix smart value checks * fix style issues * Make sure port data is stored the same way as before for Graphite and OpenTSDB Add ifIndex tag to all to be compatible * Missed rrdtool_tune() call * Test update WIP * OpenTSDB now includes tags * fix style
2020-03-16 14:17:58 +00:00
use LibreNMS\Data\Store\Datastore;
use LibreNMS\Util\Debug;
Refactored and update Location Geocoding (#9359) - Fix location so it is a regular database relation (this allows multiple devices to be accurately linked to one location and saves api calls) - Parse coordinates from the location more consistently - Add settings to webui - ~~Used [PHP Geocoder](http://geocoder-php.org/), which has lots of backends and is well tested. (also includes reverse and geoip)~~ - Google Maps, Bing, Mapquest, and OpenStreetMap supported initially. - Default to OpenStreetMap, which doesn't require a key. They will liberally hand out bans if you exceed 1 query per second though. - All other Geocoding APIs require an API key. (Google requires a credit card on file, but seems to be the most accurate) - Update all (I think) sql queries to handle the new structure - Remove final vestiges of override_sysLocation as a device attribute - Update existing device groups and rules in DB - Tested all APIs with good/bad location, no/bad/good key, and no connection. - Cannot fix advanced queries that use location This blocks #8868 DO NOT DELETE THIS TEXT #### Please note > Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting. - [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/) #### Testers If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926` After you are done testing, you can remove the changes with `./scripts/github-remove`. If there are schema changes, you can ask on discord how to revert.
2018-11-28 22:49:18 +00:00
$init_modules = ['polling', 'alerts', 'laravel'];
require __DIR__ . '/includes/init.php';
$poller_start = microtime(true);
2021-11-18 01:23:55 +00:00
Log::setDefaultDriver('console');
echo Config::get('project_name') . " Poller\n";
2015-07-13 18:10:26 +00:00
$options = getopt('h:m:i:n:r::d::v::a::f::q');
2015-07-13 18:10:26 +00:00
2019-01-08 03:26:57 +00:00
if (isset($options['h'])) {
if ($options['h'] == 'odd') {
$options['n'] = '1';
$options['i'] = '2';
} elseif ($options['h'] == 'even') {
$options['n'] = '0';
$options['i'] = '2';
} elseif ($options['h'] == 'all') {
$where = ' ';
$doing = 'all';
} elseif ($options['h']) {
if (is_numeric($options['h'])) {
2020-09-21 13:59:34 +00:00
$where = 'AND `device_id` = ' . $options['h'];
2019-01-08 03:26:57 +00:00
$doing = $options['h'];
} else {
2019-01-08 03:26:57 +00:00
if (preg_match('/\*/', $options['h'])) {
$where = "AND `hostname` LIKE '" . str_replace('*', '%', $options['h']) . "'";
2019-01-08 03:26:57 +00:00
} else {
$where = "AND `hostname` = '" . $options['h'] . "'";
2019-01-08 03:26:57 +00:00
}
$doing = $options['h'];
}
2015-07-13 18:10:26 +00:00
}
}
2015-07-13 18:10:26 +00:00
if (isset($options['i']) && $options['i'] && isset($options['n'])) {
$where = true;
// FIXME
$query = 'SELECT * FROM (SELECT @rownum :=0) r,
2015-07-13 18:10:26 +00:00
(
SELECT @rownum := @rownum +1 AS rownum, `devices`.*
2015-07-13 18:10:26 +00:00
FROM `devices`
WHERE `disabled` = 0
ORDER BY `device_id` ASC
) temp
WHERE MOD(temp.rownum, ' . $options['i'] . ') = ' . $options['n'] . ';';
$doing = $options['n'] . '/' . $options['i'];
}
2019-01-08 03:26:57 +00:00
if (empty($where)) {
2015-07-13 18:10:26 +00:00
echo "-h <device id> | <device hostname wildcard> Poll single device\n";
echo "-h odd Poll odd numbered devices (same as -i 2 -n 0)\n";
echo "-h even Poll even numbered devices (same as -i 2 -n 1)\n";
echo "-h all Poll all devices\n\n";
2015-07-13 18:10:26 +00:00
echo "-i <instances> -n <number> Poll as instance <number> of <instances>\n";
echo " Instances start at 0. 0-3 for -n 4\n\n";
2015-07-13 18:10:26 +00:00
echo "Debugging and testing options:\n";
echo "-r Do not create or update RRDs\n";
echo "-f Do not insert data into InfluxDB\n";
echo "-p Do not insert data into Prometheus\n";
echo "-d Enable debugging output\n";
echo "-v Enable verbose debugging output\n";
echo "-m Specify module(s) to be run. Comma separate modules, submodules may be added with /\n";
2015-07-13 18:10:26 +00:00
echo "\n";
echo "No polling type specified!\n";
exit;
}
if (Debug::set(isset($options['d']), false) || isset($options['v'])) {
$versions = version_info();
echo <<<EOH
===================================
Version info:
Commit SHA: {$versions['local_sha']}
Commit Date: {$versions['local_date']}
DB Schema: {$versions['db_schema']}
PHP: {$versions['php_ver']}
MySQL: {$versions['mysql_ver']}
RRDTool: {$versions['rrdtool_ver']}
SNMP: {$versions['netsnmp_ver']}
==================================
EOH;
2015-07-13 18:10:26 +00:00
echo "DEBUG!\n";
if (isset($options['v'])) {
Debug::setVerbose();
}
\LibreNMS\Util\OS::updateCache(true); // Force update of OS Cache
}
// If we've specified modules with -m, use them
$module_override = parse_modules('poller', $options);
Refactor Datastores to allow future improvements. OpenTSDB Tags. (#11283) * Datastores to object oriented code, using the Laravel IoC container Change instantiation better DI move OpenTSDB Small re-orgs remove unused stuff Fix graphs and other scripts Use DI for all except rrd fix up connection error handling Add tests, fix up a "few" things Add Config::forget() Style fixes Don't reference legacy code remove accidental code paste Add datastores phpunit groups some tests * rebase fixes * some test fixes * shorter tests * shorter tests * Don't except when rrdtool can't be started. * restore tests * fix rrd tests * fix iterable change upstream * fix isValidDataset * fix invalid data bug * fix mysql incorrect ds * fix issue with data that is too long * use regular data_update() * Use log facade * OpenTSDB mis-ordered arguments fix * Making a singleton with different options makes different singletons. Just use the global config settings to disable datastores. * only filter tags for datastores that won't it don't modify the tags permanently * Update copyrights to include original authors. * Stats for all datastores * Fix mysql sends different rrd / other ds names * fix snmp last stats not initialized remove unused function * remove unused function and move single use function closer to its use * InfluxDB does not need to update null or U values. Skip write if all fields are empty * Fix smart value checks * fix style issues * Make sure port data is stored the same way as before for Graphite and OpenTSDB Add ifIndex tag to all to be compatible * Missed rrdtool_tune() call * Test update WIP * OpenTSDB now includes tags * fix style
2020-03-16 14:17:58 +00:00
$datastore = Datastore::init($options);
2015-07-13 18:10:26 +00:00
echo "Starting polling run:\n\n";
$polled_devices = 0;
$unreachable_devices = 0;
if (! isset($query)) {
$query = "SELECT * FROM `devices` WHERE `disabled` = 0 $where ORDER BY `device_id` ASC";
}
2015-07-13 18:10:26 +00:00
foreach (dbFetch($query) as $device) {
DeviceCache::setPrimary($device['device_id']);
if (! poll_device($device, $module_override)) {
$unreachable_devices++;
}
// Update device_groups
echo "### Start Device Groups ###\n";
$dg_start = microtime(true);
2021-11-18 01:23:55 +00:00
$group_changes = Action::execute(UpdateDeviceGroupsAction::class);
2020-09-21 13:59:34 +00:00
d_echo('Groups Added: ' . implode(',', $group_changes['attached']) . PHP_EOL);
d_echo('Groups Removed: ' . implode(',', $group_changes['detached']) . PHP_EOL);
echo '### End Device Groups, runtime: ' . round(microtime(true) - $dg_start, 4) . "s ### \n\n";
echo "#### Start Alerts ####\n";
$rules = new AlertRules();
$rules->runRules($device['device_id']);
echo "#### End Alerts ####\r\n";
2015-07-13 18:10:26 +00:00
$polled_devices++;
}
$poller_end = microtime(true);
$poller_run = ($poller_end - $poller_start);
2015-07-13 18:10:26 +00:00
$poller_time = substr($poller_run, 0, 5);
$string = $argv[0] . " $doing " . date(Config::get('dateformat.compact')) . " - $polled_devices devices polled in $poller_time secs";
d_echo("$string\n");
if (! isset($options['q'])) {
echo PHP_EOL;
app(\App\Polling\Measure\MeasurementManager::class)->printStats();
}
logfile($string);
Refactor Datastores to allow future improvements. OpenTSDB Tags. (#11283) * Datastores to object oriented code, using the Laravel IoC container Change instantiation better DI move OpenTSDB Small re-orgs remove unused stuff Fix graphs and other scripts Use DI for all except rrd fix up connection error handling Add tests, fix up a "few" things Add Config::forget() Style fixes Don't reference legacy code remove accidental code paste Add datastores phpunit groups some tests * rebase fixes * some test fixes * shorter tests * shorter tests * Don't except when rrdtool can't be started. * restore tests * fix rrd tests * fix iterable change upstream * fix isValidDataset * fix invalid data bug * fix mysql incorrect ds * fix issue with data that is too long * use regular data_update() * Use log facade * OpenTSDB mis-ordered arguments fix * Making a singleton with different options makes different singletons. Just use the global config settings to disable datastores. * only filter tags for datastores that won't it don't modify the tags permanently * Update copyrights to include original authors. * Stats for all datastores * Fix mysql sends different rrd / other ds names * fix snmp last stats not initialized remove unused function * remove unused function and move single use function closer to its use * InfluxDB does not need to update null or U values. Skip write if all fields are empty * Fix smart value checks * fix style issues * Make sure port data is stored the same way as before for Graphite and OpenTSDB Add ifIndex tag to all to be compatible * Missed rrdtool_tune() call * Test update WIP * OpenTSDB now includes tags * fix style
2020-03-16 14:17:58 +00:00
Datastore::terminate();
2015-07-13 18:10:26 +00:00
// Remove this for testing
// print_r(get_defined_vars());
if ($polled_devices === $unreachable_devices) {
exit(6);
}
exit(0);