Commit Graph

169 Commits

Author SHA1 Message Date
Tony Murray
e4451714e2
version and git helper improvements (#14412)
* Fix up version and git helpers
Improve method names
Move all git calls into the git helper
Allow runtime and external cache of results where appropriate
Consolidate version headers for discovery, poller, and validate

* Style fixes

* improve consistency in git calls

* fix style

* don't send name inconsistently

* Improve database versions

* No need to cache Version it is not used more than once currently.
2022-10-02 00:41:56 -05:00
Jellyfrog
670f964e98
Validate: use "database version" instead of "mysql version" (#14158)
* Validate: use "database version" instead of "mysql version"

* wip

* wip
2022-07-29 11:36:18 -05:00
Tony Murray
c79b187d72
Poller rewrite (Try 2) (#13525)
* core WIP

* try to finish up

* trim space too
and a couple of cleanups

* update test data

* put escapes back

* another net-snmp difference

* correct copy paste error

* WIP

* Use new code YAY

* a tiny bit more

* Kind of working

* Handle manual modules correctly

* convert core to modern module

* Only save metrics if modules is not overridden

* correct module exists check

* database error handling

* debug handling

* restore bad changes

* Introduce Actions
 RunAlertRulesAction
 UpdateDeviceGroupsAction

* tweaks to output

* Fix some issues in outside code

* Style fixes

* fixes to module status checks

* typehints!

* Use logger only and DI

* OS module not named correctly

* Work on quiet output a bit more

* generically don't change output when disabling debug if the driver is already stack

* Fix missing $device variable for legacy os polling
Fix missing dbFacile functions when no legacy modules polled in RunAlertRulesAction

* restore legacy os module shim

* use the new poller code for tests

* PollingDevice event

* Fix some issues and enable/disable error reporting around legacy modules

* typehints

* fully update baseline

* Use Process for version commands so we don't leak debug output.

* don't detect rrdtool version in ci every time

* style fixes

* Warning fixes

* more fixes

* re-update baseline

* remove diff noise

* fix up alerts

* Catch exceptions in device ip lookup

* Revert accidental snmp.inc.php poller target change
(should have been ?: not ??)
2021-11-17 19:23:55 -06:00
Tony Murray
f8764ddcb5
Revert "Poller command rewrite (#13414)" (#13524)
This reverts commit 1752d1efd4.
2021-11-16 21:40:29 -06:00
Tony Murray
1752d1efd4
Poller command rewrite (#13414)
* core WIP

* try to finish up

* trim space too
and a couple of cleanups

* update test data

* put escapes back

* another net-snmp difference

* correct copy paste error

* WIP

* Use new code YAY

* a tiny bit more

* Kind of working

* Handle manual modules correctly

* convert core to modern module

* Only save metrics if modules is not overridden

* correct module exists check

* database error handling

* debug handling

* restore bad changes

* Introduce Actions
 RunAlertRulesAction
 UpdateDeviceGroupsAction

* tweaks to output

* Fix some issues in outside code

* Style fixes

* fixes to module status checks

* typehints!

* Use logger only and DI

* OS module not named correctly

* Work on quiet output a bit more

* generically don't change output when disabling debug if the driver is already stack

* Fix missing $device variable for legacy os polling
Fix missing dbFacile functions when no legacy modules polled in RunAlertRulesAction

* restore legacy os module shim

* use the new poller code for tests

* PollingDevice event

* Fix some issues and enable/disable error reporting around legacy modules

* typehints

* fully update baseline

* Use Process for version commands so we don't leak debug output.

* don't detect rrdtool version in ci every time

* style fixes

* Warning fixes

* more fixes

* re-update baseline

* remove diff noise

* fix up alerts
2021-11-16 16:59:46 -06:00
Tony Murray
8d5bc3fc41
Use Measurements for all statistic collection (#13333)
* Use Measurements for all statistic collection

* fix space

* Use colors
cleanup

* fix style

* manually fix license notice

* add return times and new line

* add return times and new line

* fix mistake in copyright template

* fix style
2021-10-06 17:09:54 -05:00
Tony Murray
d8d0ccd03d
Common contexts for polling (#13158)
* Common contexts for polling
Don't store contexts on the device array, use the device model
Also should be easier to add different contexts later.

* rename variables for consistency
2021-08-23 01:22:28 -05:00
Tony Murray
61c89794e4
Remove debug globals (#12811)
* Remove $debug global
and $vdebug global
makes these variables more accessible and protects from collisions.

* the on boot set sends application as the first parameter, just handle that

* Relocate other debug related functions

* Log debug to stdout

* Wrong output

* remove stupid constants

* Fix lint and style issues
2021-04-29 22:42:18 -05:00
Tony Murray
1c379dcd05
Remove legacy function calls (#12651)
* massive inlines

* fix style and wtf

* remove rrdtool.inc.php include

* fix CommonFunctions namespace issues

* looking for missing class space, fix undefined class issues

* style fixes
2021-03-29 00:25:30 +02:00
Tony Murray
0d61deb8f0
Remove perf_times table (#12517)
the data is never used... This type of data is better to store in rrd anyway
2021-02-09 12:47:05 -06:00
Jellyfrog
20b4215204
Swich links to https (#12511)
* Switch librenms links to https

* Convert librenms links in comments

* Switch gnu.org url to https

* switch php urls to https
2021-02-09 00:29:04 +01:00
Jellyfrog
29f45ca352
Apply fixes from StyleCI (#12124) 2020-09-21 15:59:34 +02:00
Jellyfrog
0d56bbd946
Apply fixes from StyleCI (#12123) 2020-09-21 15:51:53 +02:00
Jellyfrog
77c531527c
Apply fixes from StyleCI (#12117)
* Apply fixes from StyleCI

* Disable style check
2020-09-21 14:54:51 +02:00
Albert Siersema
4f859cee65
Fix dynamic group membership rule for devices.status equal 0 (#11699)
* Fix dynamic group membership rule for devices.status equal 0

* Fix dynamic group membership rule for devices.status equal 0 - moved code to right before running alerts
2020-05-30 18:23:31 -05:00
Tony Murray
d5a52ca4eb
Fix various issues with loading os definitions (#11640)
* Ping only device doesn't display
if os was set to something, ping os wasn't loaded and we try to get overview graphs from it.

* Fix snmp_disable device page load error
When other os is set.

* Revamp os setting loading
the only safe way to access is Config::getOsSetting()

* Remove getOsSetting fallback behavior
Most instances don't use it and it can have unexpected results Config::getOsSetting('blah', 'group') == 'librenms'

* refactor and remove unneeded load_os/loadOs calls now since getOsSetting automatically loads it.

* restore unix overview graphs, they are different
small cleanups

* fix
2020-05-19 14:35:32 -05:00
Tony Murray
df16de9d2f
Refactor Datastores to allow future improvements. OpenTSDB Tags. (#11283)
* Datastores to object oriented code, using the Laravel IoC container

Change instantiation
better DI
move OpenTSDB

Small re-orgs

remove unused stuff

Fix graphs and other scripts

Use DI for all except rrd

fix up connection error handling

Add tests, fix up a "few" things
Add Config::forget()

Style fixes

Don't reference legacy code

remove accidental code paste

Add datastores phpunit groups

some tests

* rebase fixes

* some test fixes

* shorter tests

* shorter tests

* Don't except when rrdtool can't be started.

* restore tests

* fix rrd tests

* fix iterable change upstream

* fix isValidDataset

* fix invalid data bug

* fix mysql incorrect ds

* fix issue with data that is too long

* use regular data_update()

* Use log facade

* OpenTSDB mis-ordered arguments fix

* Making a singleton with different options makes different singletons.  Just use the global config settings to disable datastores.

* only filter tags for datastores that won't it don't modify the tags permanently

* Update copyrights to include original authors.

* Stats for all datastores

* Fix mysql sends different rrd / other ds names

* fix snmp last stats not initialized
remove unused function

* remove unused function and move single use function closer to its use

* InfluxDB does not need to update null or U values.
Skip write if all fields are empty

* Fix smart value checks

* fix style issues

* Make sure port data is stored the same way as before for Graphite and OpenTSDB
Add ifIndex tag to all to be compatible

* Missed rrdtool_tune() call

* Test update WIP

* OpenTSDB now includes tags

* fix style
2020-03-16 09:17:58 -05:00
Tony Murray
7fe895bd08
Revert "Refactor datastores to classes (#9179)" (#11273)
This reverts commit e5dad7a64e.
2020-03-11 07:52:52 -05:00
Tony Murray
e5dad7a64e
Refactor datastores to classes (#9179)
* Datastores to object oriented code, using the Laravel IoC container

Change instantiation
better DI
move OpenTSDB

Small re-orgs

remove unused stuff

Fix graphs and other scripts

Use DI for all except rrd

fix up connection error handling

Add tests, fix up a "few" things
Add Config::forget()

Style fixes

Don't reference legacy code

remove accidental code paste

Add datastores phpunit groups

some tests

* rebase fixes

* some test fixes

* shorter tests

* shorter tests

* Don't except when rrdtool can't be started.

* restore tests

* fix rrd tests

* fix iterable change upstream

* fix isValidDataset

* fix invalid data bug

* fix mysql incorrect ds

* fix issue with data that is too long

* use regular data_update()

* Use log facade

* OpenTSDB mis-ordered arguments fix

* Making a singleton with different options makes different singletons.  Just use the global config settings to disable datastores.

* only filter tags for datastores that won't it don't modify the tags permanently

* Update copyrights to include original authors.

* Stats for all datastores

* Fix mysql sends different rrd / other ds names

* fix snmp last stats not initialized
remove unused function

* remove unused function and move single use function closer to its use

* InfluxDB does not need to update null or U values.
Skip write if all fields are empty

* Fix smart value checks

* fix style issues
2020-03-10 19:08:00 -05:00
Tony Murray
dec9a498ee Updated device cache support (#10795)
* Device Cache
also some additional device related models

* attribs from cache

* replace common attribute functions

* remove legacy cache usage
tidy up some collection manipulation
remove some unused or single use functions

* cleanup some items

* always return a device, to prevent bugs

* clear device cache when testing after each test

* fix double assignment

* Clean up function to take advantage of null object
2019-11-14 21:56:06 +00:00
Tony Murray
699aa8a042
Consolidate configuration settings and implement dynamic webui settings (#9809)
* initial db/defaults dump

* numeric values

* Remove $config['time']

* Use config_definitions.json

* try setting definition as a complete array

* a little more

* fix format

* WIP converting dynamic setting pages

* rewriting the webui

* remove legacy and add translations

* finish email section

* improve navigation
update js a bit

* Import the remaining existing settings

* Update backend still some wip
migration

* fix config loading (db not overriding defaults)

* some cleanup

* more array collapsing

* update settings

* Settings search

* add purge settings to ui, order groups and sections

* collapse more arrays

* Auth definitions WIP

* remove needless slash escapes

* add warning to json

* moving settings from defaults.inc

* remove slow_statistics

* Move rrdtool settings to the poller section

* Use translations more consistently

* removing more settings from defaults.inc.php

* show setting name in tooltip

* allow help text to be shown on mobile

* disable settings that are set in config.php

* Implement undo and reset to default.

* Vue.js functional

* Vue.js WIP

* Implement tabs component

* accordion WIP

* lodash ver update

* two items

* Accordion somewhat working

* hash navigation

* Refine Accordion

* Fix up tab styling a bit

* Cleaner tab selected property
Hide html while loading

* spinner?

* Icon support
property for accordion to active

* WIP

* Tabs/Sections from ajax

* Setting Component skeletons

* Dynamic Component resolution

* Basic functionality

* toggle class

* Refactor components

* translate tabs and accordions

* simple array attempt

* improve readonly tooltip

* array styling

* array value editing

* organize snmp info

* Handle initial tab/section in url

* Use Laravel to parse tab/section, dump old

* Draggable array entries

* v-tooltip, for clickable (and touch) help tooltips
disable draggable

* Navigation WIP

* Navigation WIP

* groups computed

* filter settings

* fix event reference

* vue.js i18n initial

* missing description = just setting name

* en fallback

* tidy up the language support and js generation

* persist value to db

* fix issue with 0

* Delete settings from DB instead of setting them to default

* ldap-groups
fixup style

* Default dashboard selection

* fix array of options instead of object

* allow custom validation for settings

* translate options in SettingSelect

* SNMP v3 WIP

* fix setting arrays

* Split persist out of set

* Hook up events for SNMP v3 Auth
correct Config::persist behaviour with arrays

* dependent settings (primitive for now)
actually update the settings values in the LibrenmsSettings component

* more complex "when" behaviour

* remove un-needed seeder

* add poller_modules.cisco-qfp

* remove en.json (disable warning)

* don't set default for log_dir or log_file, otherwise it won't be processed correctly

* Fix module order
add some missing settings

* more config corrections

* correct graphs
correct loading values set to null (although it should have no difference)
remove project_name_version

* Add nfsen settings.  Docs are very confusing, so might have flubbed something
remove option for array definition of select option as numeric indexes messes it up

* Correct more upstream config differences

* Config cleanup after a bunch of merges.

* Fixes

* add version tags to js/css files
remove old js

* Print out full settings list read-only

* Add http_proxy setting
fix indents in config_definitions.json

* repeaters default is 0 (aka 20)

* cleanups

* rewrite the dynamic config docs

* add language docs

* Don't show snmp v3 auth add/remove if disabled by config.php
2019-10-16 21:22:05 +00:00
Heath Barnhart
851994cfb8 Alert Subsys to OOP and SNMPTraps trigger Alert Rules (#9765)
* Changed snmptrap.php and handlers to allow for alerts.

* Moved RunRules() to Dispatcher.php

* Modified includes, passes pre-commit tests locally.

* Converted RunRules to OO, fixed formatting in files found by TravisCI

* rebase, covert alert subsy to oop

* Modified init.php to call new Alert subsystem

* fixed dependencies

* Resolved undefined functions

* Fixed PingCheck.php not triggering

* Changed runAlerts to __construct

* Fixes
Can't have Config inside empty()
incorrectly named function (my fault)
poller.php missing shebang
organize imports
2019-07-18 08:36:02 -05:00
Tony Murray
f3ba8947f7
Use Config helper (#10339)
remove usage of global variable
2019-06-23 00:29:12 -05:00
Tony Murray
d303174e7a
fix poller.php with missing -h (#9621) 2019-01-07 21:26:57 -06:00
Tony Murray
3e35ee0e7d Refactored and update Location Geocoding (#9359)
- Fix location so it is a regular database relation (this allows multiple devices to be accurately linked to one location and saves api calls)
- Parse coordinates from the location more consistently
- Add settings to webui
- ~~Used [PHP Geocoder](http://geocoder-php.org/), which has lots of backends and is well tested. (also includes reverse and geoip)~~
- Google Maps, Bing, Mapquest, and OpenStreetMap supported initially.
- Default to OpenStreetMap, which doesn't require a key.  They will liberally hand out bans if you exceed 1 query per second though.
- All other Geocoding APIs require an API key. (Google requires a credit card on file, but seems to be the most accurate)
- Update all (I think) sql queries to handle the new structure
- Remove final vestiges of override_sysLocation as a device attribute
- Update existing device groups and rules in DB
- Tested all APIs with good/bad location, no/bad/good key, and no connection.
- Cannot fix advanced queries that use location

This blocks #8868

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
After you are done testing, you can remove the changes with `./scripts/github-remove`.  If there are schema changes, you can ask on discord how to revert.
2018-11-28 22:49:18 +00:00
Tony Murray
eeb3d58f5b Improved Logging and Debugging (#8870)
Use Log facility when Laravel is booted.
Update init.php so we can easily boot Laravel for CLI scripts. (and just Eloquent, but that may go away)
Move all debug setup into set_debug() function and use that across all scripts.
Log Laravel database queries.
Send debug output to librenms log file when enabling debug in the webui.
Allow for colorized Log CLI output. (currently will leave % tags in log file output)

** Needs testing and perhaps tweaking still.

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-07-13 23:08:00 +01:00
Tony Murray
846cf11331 Support specifying submodules in poller and discovery for debug (#8896)
Mostly useful for debugging

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-07-12 21:30:39 +01:00
Tony Murray
0ba76e6d62 New python service for poller, discovery + more (#8455)
Currently has a file handle leak (and will eventually run out of handles) related to the self update process.

Either need to fix that or rip out self-update and leave that up to cron or something.


DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-06-30 12:19:49 +01:00
cchance27
09d6b2fd54 feature: Added Prometheus PushGateway metric support (#8437)
* Added Prometheus PushGateway support

* fixed formatting

* fix set_curl_proxy to correct variable

* add documentation

* Update Metric-Storage.md

* Update mkdocs.yml
2018-04-13 15:46:19 +01:00
Neil Lathwood
ecbbc0e3e5 feature: Clear OS defs cache on poller/discovery debug (#8061) 2018-01-09 10:51:26 -06:00
Tony Murray
51ba934e11 feature: Added web validation (#7474)
* refactor: convert validations to modules to prep for gui integration

* accidentally dropped, schema update

* fix accidental output to webui

* mention discovery-wrapper.py and re-arrange.

* refine some fix statements

* rename the Config validation group to Configuration.

* fix some scrutinizer issues
remove as many local functions from validator.php as possible
move extensions from pre-check
remove duplicate timezone check
looks like there is some db schema differences between mariadb 10.1 and 10.2, investigating

* mariadb version diff take2

* Check schema version first for database.
Remove stop to go back to command line for install docs.
Add helpful link when there is no devices added to /addhost

* fix incorrect validation for empty string defaults

* Fix style

* Add additional file permissions checks

* catch exception and fail for invalid timezone
Change visuals a bit.

* add php version warning

* fix space
2017-10-26 07:56:09 +01:00
Neil Lathwood
d49e893a0a move version info into debug to save exec calls under normal use (#6407) 2017-04-22 08:10:15 -05:00
Falk Stern
affe5090f4 feature: Send collected data to graphite server (#6201)
* Initial support for graphite

* Enable graphite include

* fixed typos.

* Fixed port naming to graphite metric names

* Added documentation for Graphite

* added documentation for graphite

* fixed style

* replace / with _ in interface names for graphite

* sets default graphite port

* adds rrd-name tags to the metric, otherwise metrics get lost

* add suggested storage schema for graphite

* add whitespace

* Updated to show some output

* bad english
2017-03-22 10:35:44 -05:00
Neil Lathwood
9ab4c1f4ee fix: restore usage of -i -n in polling (#5228) 2016-12-22 21:33:51 +00:00
Neil Lathwood
bcae14c7d9 Stop running two queries for the same thing (#5177) 2016-12-16 12:15:13 -06:00
Tony Murray
db240cb4eb refactor: Collection and output of db and snmp stats (#5049) 2016-11-23 06:57:19 +00:00
Tony Murray
9a33464c52 refactor: Centralize includes and initialization (#4991) 2016-11-21 20:12:59 +00:00
Neil Lathwood
d86b39395a refactor: Small poller improvements, removes unecessary queries / execs (#4741) 2016-10-14 20:03:26 -05:00
Neil Lathwood
8b68429a06 Tidy up alert output info for poller cli (#4578) 2016-09-25 10:01:58 -05:00
Neil Lathwood
0ee1fac5da fix: Small database query improvements in poller.php (#4512) 2016-09-21 12:56:42 -05:00
Tony Murray
d2193f76ca feature: allow scripts to be run from any working directory (#4437)
* feature: allow validate.php to be run from any working directory

* remove redundant realpath() call

* re-add realpath() to remove symbolic links

* realpath() isn't needed...

* chdir() in all php scripts in ./ and ./scripts/

* update-sql.php clearly is not used, as it was broken.

* Change some scripts to executable
Remove extra chdir() call in snmp-scan.php

* Missed console-ui.php
Kind of fixed console-ui.php help output

* Re-add newline
2016-09-14 16:53:04 +01:00
Neil Lathwood
7dec670494 refactor: Updated a lot of references to LibreNMS - copyright still left intact (#4339) 2016-09-08 08:12:23 -05:00
Tony Murray
abc6a5b799 PSR-2 Final cleanup (#4247)
refactor: Final PSR2 cleanup
2016-08-28 23:32:55 +01:00
Tony Murray
200e670315 Fix defunct process issues: create Proc class (#4210)
* Proc class
Encapsulate processes in a class, when they go out of scope, we can use the __destruct() process to make sure the process terminates.

* Fix rrdtool_last checks failing

* Don't close rrdtool in rrd_graph
Try to start rrdtool process if they haven't been started yet
Remove some extra debug output

* phpdocs added
several cleanups
should still be functioning the same
Only open one  rrdtool process for graph.php
2016-08-26 07:50:29 +01:00
Tony Murray
e80a385b7f RRDtool dual process (remote creation/check) (#4104)
* Prepare for dual rrdtool processes
Disabled at this time
Split out rrdtool version checks into rrdtool_create_command()
Tests for rrdtool_create_command()
Fixes a few small issues.

* Enable dual process and remote rrd check/creation

* remove full path for remote commands

* Doc updates
minor fix to rrdtool_tune()

* Set up bootstrap for phpunit
Fix issues with FileExistsException

* Attempt to fix phpunit

* Fix classloader and bootstrap to use full paths only (not depending on $config)

* Fix phpunit tests, config.php does not exist.
2016-08-22 16:32:05 +01:00
laf
b673432058 Added support for nets-exclude in snmp-scan 2016-08-12 21:39:53 +01:00
Tony Murray
846c5ab0c9 Add date to git version info
valdiate: local tz, rfc2822
poller: iso 8601
about: iso 8601 -> browser local
Also, do not fetch github info on about page, it isn't used.
2016-07-06 13:17:13 -05:00
Sawachier
ca9caca264 Update poller.php 2016-04-26 15:30:07 +02:00
Sawachier
1fb1f4473f Update poller.php
";" at the end of the query
2016-04-21 11:46:32 +02:00
HenocKA
111c588816 poller - adding vrf-cisco-lite in device 2016-04-20 00:52:56 +02:00