librenms/poller.php

185 lines
5.7 KiB
PHP
Raw Normal View History

#!/usr/bin/env php
<?php
/**
* poller.php
*
* -Description-
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*
* @package LibreNMS
* @subpackage poller
* @copyright (C) 2006 - 2012 Adam Armstrong
* Modified 4/17/19
* @author Heath Barnhart <hbarnhart@kanren.net>
*/
use LibreNMS\Config;
use LibreNMS\Alert\AlertRules;
Refactor Datastores to allow future improvements. OpenTSDB Tags. (#11283) * Datastores to object oriented code, using the Laravel IoC container Change instantiation better DI move OpenTSDB Small re-orgs remove unused stuff Fix graphs and other scripts Use DI for all except rrd fix up connection error handling Add tests, fix up a "few" things Add Config::forget() Style fixes Don't reference legacy code remove accidental code paste Add datastores phpunit groups some tests * rebase fixes * some test fixes * shorter tests * shorter tests * Don't except when rrdtool can't be started. * restore tests * fix rrd tests * fix iterable change upstream * fix isValidDataset * fix invalid data bug * fix mysql incorrect ds * fix issue with data that is too long * use regular data_update() * Use log facade * OpenTSDB mis-ordered arguments fix * Making a singleton with different options makes different singletons. Just use the global config settings to disable datastores. * only filter tags for datastores that won't it don't modify the tags permanently * Update copyrights to include original authors. * Stats for all datastores * Fix mysql sends different rrd / other ds names * fix snmp last stats not initialized remove unused function * remove unused function and move single use function closer to its use * InfluxDB does not need to update null or U values. Skip write if all fields are empty * Fix smart value checks * fix style issues * Make sure port data is stored the same way as before for Graphite and OpenTSDB Add ifIndex tag to all to be compatible * Missed rrdtool_tune() call * Test update WIP * OpenTSDB now includes tags * fix style
2020-03-16 14:17:58 +00:00
use LibreNMS\Data\Store\Datastore;
Refactored and update Location Geocoding (#9359) - Fix location so it is a regular database relation (this allows multiple devices to be accurately linked to one location and saves api calls) - Parse coordinates from the location more consistently - Add settings to webui - ~~Used [PHP Geocoder](http://geocoder-php.org/), which has lots of backends and is well tested. (also includes reverse and geoip)~~ - Google Maps, Bing, Mapquest, and OpenStreetMap supported initially. - Default to OpenStreetMap, which doesn't require a key. They will liberally hand out bans if you exceed 1 query per second though. - All other Geocoding APIs require an API key. (Google requires a credit card on file, but seems to be the most accurate) - Update all (I think) sql queries to handle the new structure - Remove final vestiges of override_sysLocation as a device attribute - Update existing device groups and rules in DB - Tested all APIs with good/bad location, no/bad/good key, and no connection. - Cannot fix advanced queries that use location This blocks #8868 DO NOT DELETE THIS TEXT #### Please note > Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting. - [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/) #### Testers If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926` After you are done testing, you can remove the changes with `./scripts/github-remove`. If there are schema changes, you can ask on discord how to revert.
2018-11-28 22:49:18 +00:00
$init_modules = ['polling', 'alerts', 'laravel'];
require __DIR__ . '/includes/init.php';
$poller_start = microtime(true);
Consolidate configuration settings and implement dynamic webui settings (#9809) * initial db/defaults dump * numeric values * Remove $config['time'] * Use config_definitions.json * try setting definition as a complete array * a little more * fix format * WIP converting dynamic setting pages * rewriting the webui * remove legacy and add translations * finish email section * improve navigation update js a bit * Import the remaining existing settings * Update backend still some wip migration * fix config loading (db not overriding defaults) * some cleanup * more array collapsing * update settings * Settings search * add purge settings to ui, order groups and sections * collapse more arrays * Auth definitions WIP * remove needless slash escapes * add warning to json * moving settings from defaults.inc * remove slow_statistics * Move rrdtool settings to the poller section * Use translations more consistently * removing more settings from defaults.inc.php * show setting name in tooltip * allow help text to be shown on mobile * disable settings that are set in config.php * Implement undo and reset to default. * Vue.js functional * Vue.js WIP * Implement tabs component * accordion WIP * lodash ver update * two items * Accordion somewhat working * hash navigation * Refine Accordion * Fix up tab styling a bit * Cleaner tab selected property Hide html while loading * spinner? * Icon support property for accordion to active * WIP * Tabs/Sections from ajax * Setting Component skeletons * Dynamic Component resolution * Basic functionality * toggle class * Refactor components * translate tabs and accordions * simple array attempt * improve readonly tooltip * array styling * array value editing * organize snmp info * Handle initial tab/section in url * Use Laravel to parse tab/section, dump old * Draggable array entries * v-tooltip, for clickable (and touch) help tooltips disable draggable * Navigation WIP * Navigation WIP * groups computed * filter settings * fix event reference * vue.js i18n initial * missing description = just setting name * en fallback * tidy up the language support and js generation * persist value to db * fix issue with 0 * Delete settings from DB instead of setting them to default * ldap-groups fixup style * Default dashboard selection * fix array of options instead of object * allow custom validation for settings * translate options in SettingSelect * SNMP v3 WIP * fix setting arrays * Split persist out of set * Hook up events for SNMP v3 Auth correct Config::persist behaviour with arrays * dependent settings (primitive for now) actually update the settings values in the LibrenmsSettings component * more complex "when" behaviour * remove un-needed seeder * add poller_modules.cisco-qfp * remove en.json (disable warning) * don't set default for log_dir or log_file, otherwise it won't be processed correctly * Fix module order add some missing settings * more config corrections * correct graphs correct loading values set to null (although it should have no difference) remove project_name_version * Add nfsen settings. Docs are very confusing, so might have flubbed something remove option for array definition of select option as numeric indexes messes it up * Correct more upstream config differences * Config cleanup after a bunch of merges. * Fixes * add version tags to js/css files remove old js * Print out full settings list read-only * Add http_proxy setting fix indents in config_definitions.json * repeaters default is 0 (aka 20) * cleanups * rewrite the dynamic config docs * add language docs * Don't show snmp v3 auth add/remove if disabled by config.php
2019-10-16 21:22:05 +00:00
echo Config::get('project_name')." Poller\n";
2015-07-13 18:10:26 +00:00
$options = getopt('h:m:i:n:r::d::v::a::f::q');
2015-07-13 18:10:26 +00:00
2019-01-08 03:26:57 +00:00
if (isset($options['h'])) {
if ($options['h'] == 'odd') {
$options['n'] = '1';
$options['i'] = '2';
} elseif ($options['h'] == 'even') {
$options['n'] = '0';
$options['i'] = '2';
} elseif ($options['h'] == 'all') {
$where = ' ';
$doing = 'all';
} elseif ($options['h']) {
if (is_numeric($options['h'])) {
$where = "AND `device_id` = " . $options['h'];
$doing = $options['h'];
} else {
2019-01-08 03:26:57 +00:00
if (preg_match('/\*/', $options['h'])) {
$where = "AND `hostname` LIKE '" . str_replace('*', '%', mres($options['h'])) . "'";
} else {
$where = "AND `hostname` = '" . mres($options['h']) . "'";
}
$doing = $options['h'];
}
2015-07-13 18:10:26 +00:00
}
}
2015-07-13 18:10:26 +00:00
if (isset($options['i']) && $options['i'] && isset($options['n'])) {
$where = true;
// FIXME
$query = 'SELECT * FROM (SELECT @rownum :=0) r,
2015-07-13 18:10:26 +00:00
(
SELECT @rownum := @rownum +1 AS rownum, `devices`.*
2015-07-13 18:10:26 +00:00
FROM `devices`
WHERE `disabled` = 0
ORDER BY `device_id` ASC
) temp
WHERE MOD(temp.rownum, '.mres($options['i']).') = '.mres($options['n']).';';
$doing = $options['n'].'/'.$options['i'];
}
2019-01-08 03:26:57 +00:00
if (empty($where)) {
2015-07-13 18:10:26 +00:00
echo "-h <device id> | <device hostname wildcard> Poll single device\n";
echo "-h odd Poll odd numbered devices (same as -i 2 -n 0)\n";
echo "-h even Poll even numbered devices (same as -i 2 -n 1)\n";
echo "-h all Poll all devices\n\n";
2015-07-13 18:10:26 +00:00
echo "-i <instances> -n <number> Poll as instance <number> of <instances>\n";
echo " Instances start at 0. 0-3 for -n 4\n\n";
2015-07-13 18:10:26 +00:00
echo "Debugging and testing options:\n";
echo "-r Do not create or update RRDs\n";
echo "-f Do not insert data into InfluxDB\n";
echo "-p Do not insert data into Prometheus\n";
echo "-d Enable debugging output\n";
echo "-v Enable verbose debugging output\n";
echo "-m Specify module(s) to be run. Comma separate modules, submodules may be added with /\n";
2015-07-13 18:10:26 +00:00
echo "\n";
echo "No polling type specified!\n";
exit;
}
if (set_debug(isset($options['d'])) || isset($options['v'])) {
$versions = version_info();
echo <<<EOH
===================================
Version info:
Commit SHA: {$versions['local_sha']}
Commit Date: {$versions['local_date']}
DB Schema: {$versions['db_schema']}
PHP: {$versions['php_ver']}
MySQL: {$versions['mysql_ver']}
RRDTool: {$versions['rrdtool_ver']}
SNMP: {$versions['netsnmp_ver']}
==================================
EOH;
2015-07-13 18:10:26 +00:00
echo "DEBUG!\n";
if (isset($options['v'])) {
$vdebug = true;
}
update_os_cache(true); // Force update of OS Cache
}
// If we've specified modules with -m, use them
$module_override = parse_modules('poller', $options);
Refactor Datastores to allow future improvements. OpenTSDB Tags. (#11283) * Datastores to object oriented code, using the Laravel IoC container Change instantiation better DI move OpenTSDB Small re-orgs remove unused stuff Fix graphs and other scripts Use DI for all except rrd fix up connection error handling Add tests, fix up a "few" things Add Config::forget() Style fixes Don't reference legacy code remove accidental code paste Add datastores phpunit groups some tests * rebase fixes * some test fixes * shorter tests * shorter tests * Don't except when rrdtool can't be started. * restore tests * fix rrd tests * fix iterable change upstream * fix isValidDataset * fix invalid data bug * fix mysql incorrect ds * fix issue with data that is too long * use regular data_update() * Use log facade * OpenTSDB mis-ordered arguments fix * Making a singleton with different options makes different singletons. Just use the global config settings to disable datastores. * only filter tags for datastores that won't it don't modify the tags permanently * Update copyrights to include original authors. * Stats for all datastores * Fix mysql sends different rrd / other ds names * fix snmp last stats not initialized remove unused function * remove unused function and move single use function closer to its use * InfluxDB does not need to update null or U values. Skip write if all fields are empty * Fix smart value checks * fix style issues * Make sure port data is stored the same way as before for Graphite and OpenTSDB Add ifIndex tag to all to be compatible * Missed rrdtool_tune() call * Test update WIP * OpenTSDB now includes tags * fix style
2020-03-16 14:17:58 +00:00
set_debug($debug);
Refactor Datastores to allow future improvements. OpenTSDB Tags. (#11283) * Datastores to object oriented code, using the Laravel IoC container Change instantiation better DI move OpenTSDB Small re-orgs remove unused stuff Fix graphs and other scripts Use DI for all except rrd fix up connection error handling Add tests, fix up a "few" things Add Config::forget() Style fixes Don't reference legacy code remove accidental code paste Add datastores phpunit groups some tests * rebase fixes * some test fixes * shorter tests * shorter tests * Don't except when rrdtool can't be started. * restore tests * fix rrd tests * fix iterable change upstream * fix isValidDataset * fix invalid data bug * fix mysql incorrect ds * fix issue with data that is too long * use regular data_update() * Use log facade * OpenTSDB mis-ordered arguments fix * Making a singleton with different options makes different singletons. Just use the global config settings to disable datastores. * only filter tags for datastores that won't it don't modify the tags permanently * Update copyrights to include original authors. * Stats for all datastores * Fix mysql sends different rrd / other ds names * fix snmp last stats not initialized remove unused function * remove unused function and move single use function closer to its use * InfluxDB does not need to update null or U values. Skip write if all fields are empty * Fix smart value checks * fix style issues * Make sure port data is stored the same way as before for Graphite and OpenTSDB Add ifIndex tag to all to be compatible * Missed rrdtool_tune() call * Test update WIP * OpenTSDB now includes tags * fix style
2020-03-16 14:17:58 +00:00
$datastore = Datastore::init($options);
2015-07-13 18:10:26 +00:00
echo "Starting polling run:\n\n";
$polled_devices = 0;
$unreachable_devices = 0;
2015-07-13 18:10:26 +00:00
if (!isset($query)) {
$query = "SELECT * FROM `devices` WHERE `disabled` = 0 $where ORDER BY `device_id` ASC";
}
2015-07-13 18:10:26 +00:00
foreach (dbFetch($query) as $device) {
DeviceCache::setPrimary($device['device_id']);
if ($device['os_group'] == 'cisco') {
$device['vrf_lite_cisco'] = dbFetchRows("SELECT * FROM `vrf_lite_cisco` WHERE `device_id` = " . $device['device_id']);
} else {
$device['vrf_lite_cisco'] = '';
}
if (!poll_device($device, $module_override)) {
$unreachable_devices++;
}
echo "#### Start Alerts ####\n";
$rules = new AlertRules();
$rules->runRules($device['device_id']);
echo "#### End Alerts ####\r\n";
2015-07-13 18:10:26 +00:00
$polled_devices++;
}
$poller_end = microtime(true);
2015-07-13 18:10:26 +00:00
$poller_run = ($poller_end - $poller_start);
$poller_time = substr($poller_run, 0, 5);
2015-07-13 18:10:26 +00:00
if ($polled_devices) {
dbInsert(array(
'type' => 'poll',
'doing' => $doing,
'start' => $poller_start,
'duration' => $poller_time,
'devices' => $polled_devices,
'poller' => Config::get('base_url')
), 'perf_times');
}
$string = $argv[0] . " $doing " . date(Config::get('dateformat.compact')) . " - $polled_devices devices polled in $poller_time secs";
d_echo("$string\n");
if (!isset($options['q'])) {
printStats();
}
logfile($string);
Refactor Datastores to allow future improvements. OpenTSDB Tags. (#11283) * Datastores to object oriented code, using the Laravel IoC container Change instantiation better DI move OpenTSDB Small re-orgs remove unused stuff Fix graphs and other scripts Use DI for all except rrd fix up connection error handling Add tests, fix up a "few" things Add Config::forget() Style fixes Don't reference legacy code remove accidental code paste Add datastores phpunit groups some tests * rebase fixes * some test fixes * shorter tests * shorter tests * Don't except when rrdtool can't be started. * restore tests * fix rrd tests * fix iterable change upstream * fix isValidDataset * fix invalid data bug * fix mysql incorrect ds * fix issue with data that is too long * use regular data_update() * Use log facade * OpenTSDB mis-ordered arguments fix * Making a singleton with different options makes different singletons. Just use the global config settings to disable datastores. * only filter tags for datastores that won't it don't modify the tags permanently * Update copyrights to include original authors. * Stats for all datastores * Fix mysql sends different rrd / other ds names * fix snmp last stats not initialized remove unused function * remove unused function and move single use function closer to its use * InfluxDB does not need to update null or U values. Skip write if all fields are empty * Fix smart value checks * fix style issues * Make sure port data is stored the same way as before for Graphite and OpenTSDB Add ifIndex tag to all to be compatible * Missed rrdtool_tune() call * Test update WIP * OpenTSDB now includes tags * fix style
2020-03-16 14:17:58 +00:00
Datastore::terminate();
2015-07-13 18:10:26 +00:00
// Remove this for testing
// print_r(get_defined_vars());
if ($polled_devices === $unreachable_devices) {
exit(6);
}
exit(0);