Commit Graph

93 Commits

Author SHA1 Message Date
Tony Murray
49f8269262
Improved Latency graph (#15940)
* Improved Latency graph
Store loss+jitter info in rrd instead of database
New graph icmp_perf (legacy ping_perf still valid referencing part of the newer data)
Delete device_perf table

* Change loss to an area so it is more visible

* Style fixes

* Cleanups from phpstan & tests

* exit_code fix

* Remove alert usage of device_perf

* Don't use magic __get

* Add test for bulkPing
Add host to previous tests

* style fixes

* Fix issue fping error responses
2024-04-18 09:57:01 -05:00
PipoCanaja
a7a6e67e87
NAC - Improve search in WebUI - Keep Historical data (#15629)
* search improvement for PortNac

* rename func

* style

* DB migration for timestamps

* style

* rules

* fix manually db_schema

* remove vlan only search for now

* add a bool column to ident historical nac entries

* add columns to the table

* queryByOui renamed everywhere

* age value instead of delete

* style

* style

* use Illuminate\Support\Facades\DB;

* tests fix

* module_tables.yaml

* dump model instead of tests/module_tables.yaml

* tests

* testVrp

* daily

* config

* display historical NAC entries in global view

* same for NAC device page

* nac tab on device/port view

* and display the tab link if necessary

* filter by port as well

* historical data in port tab

* formatters
2024-01-24 09:15:40 +01:00
Tony Murray
4211b1c46f
Remove unused functions (#15371)
* Remove unused functions
inline other legacy functions

* Apply fixes from StyleCI

---------

Co-authored-by: StyleCI Bot <bot@styleci.io>
2023-10-04 10:17:34 -05:00
Tony Murray
12f8bb2040
MAC Vendor OUI use scheduler (#15187)
* MAC Vendor OUI use scheduler
Add command to update `lnms maintenance:fetch-ouis`
Show vendor column in tables if mac_oui.enabled is set to true
Improve scheduler validation handle non-standard install directories and systems without systemd
Add index to table to improve speed and improve mac->vendor lookup speed
Scheduled weekly with random wait to prevent stampeding herd issues for upstream
drop oui update from daily

* MAC Vendor OUI use scheduler
Add command to update `lnms maintenance:fetch-ouis`
Show vendor column in tables if mac_oui.enabled is set to true

* Lint fixes and better prefix detection

* update schema file
2023-08-03 19:29:30 -05:00
Andy Norwood
64c4650801
Add FDB table vendor search drop down (#15072)
* Add vendor search and related functions

* Add OUIDB cache for vendor lookup

* Add vendor drop down

* appy style CI changes

* Apply style CI and lint changes

* more styleCI changes

* update type hinting

* Edit mac_oui cache lock name and function

* Update MAC OUI message during daily

* Use DB for vendor lookup

* New vendor_oui table migration

* New MAC OUI to database function

* Update readbleOUI to use DB rather than cache

* Make StyleCI changes

* styleCI tweak

* Remove lock release to allow refresh timer

* change migration name to match table

* add schema dump

* update schema

* styleCI tweak
2023-07-21 11:30:13 -05:00
RobJE
65e3edd135
change rrd_prune find command to exlcude .gitignore (#15134) 2023-07-20 11:37:04 -05:00
Peter Childs
b655eb7feb
change chunk to chunkById (#15139) 2023-07-07 11:53:15 -05:00
Peter Childs
2fada03570
use lock ports_purge in ports_purge rather than syslog_purge lock (#15135) 2023-07-05 08:10:23 -05:00
Jellyfrog
68e0b3df28
Apply fixes from StyleCI (#14899)
Co-authored-by: StyleCI Bot <bot@styleci.io>
2023-03-13 22:32:22 +01:00
Tony Murray
66790e7e8f
Increase minimum version to PHP 8.1 (#14378) 2022-09-26 08:52:15 -05:00
Tony Murray
e990dfcb35
Disable plugins that have errors (#14383)
* Disable plugins that have errors
Disable plugin if a hook throws an error and set a notification
Move notification code to class, so we can access it
Clear notification when plugin is attempted to be enabled again

* fix style and lint fixes

* another lint fix and handle if property is missing
2022-09-25 22:47:58 -05:00
Jellyfrog
835eac1233
Fix some ErrorExceptions (#14241)
* Fix some ErrorExceptions

* Use Config::get default value

Co-authored-by: Tony Murray <murraytony@gmail.com>
2022-08-25 10:03:06 +02:00
Jellyfrog
d3d2aa92e1
Fix alert log clearing SQL query (#14200)
only replace one, lock_and_purge_query() already takes care of second...
2022-08-14 09:12:46 -05:00
Tony Murray
8508710e62
Stats: Collect OS distro and LibreNMS version (#14138)
* Stats: Collect OS distro and LibreNMS version

Add os distro/version and LibreNMS version to stats collection.

To view stats: https://stats.librenms.org

* style/lint fixes

* Put debug where it will work

* One call sends int, which is ok
2022-07-29 08:53:41 -05:00
Tony Murray
014213680f
Move Config loading to a service provider (#13927)
* Move Config loading to a service provider
That way other service providers can depend on it
Move various random listener registrations into the EventServiceProvider
Various startup cleanup

* Config::persist Set live variable before persisting incase db update fail

* Disable strict mode for legacy code (init.php)

* Disable debug after os test data is gathered

* remove Eloquent::boot it is never used

* remove Eloquent::version

* lint fixes

* style fixes

* there is no c_echo here
2022-04-22 19:12:07 -05:00
Jellyfrog
fdebee86b4
Implement system for user packages in composer (#13718) 2022-02-02 17:33:23 +01:00
Tony Murray
0b8b97bb68
Push Notifications (Mobile and PC) (#13277)
* Update manifest and add service worker
cleanup icons a bit

* Push notifications WIP

* navigate working

* cleanup

* acknowledge wired up

* Set VAPID keys on composer install

* Component to control notification permissions.

* Allow all user option to validate

* Enable on browser load if transport exists.

* Check for transport before showing user permissions
translations

* Documentation

* style fixes

* access via the attribute model

* fix alerting test

* update schema

* cleanup subscription on disable

* non-configurable db and table for webpush subscriptions (respect system connection)

* revert AlertTransport change
hopefully phpstan can figure it out

* phpstan fixes

* Support custom details display

* Match transport names to brand's preferred display

* less duplicate id errors

* Tests are done in Laravel code now so
remove legacy function usage... could be better, but ok

* Style fixes

* Style fixes 2

* Fix alert test

* Doc updates requires HTTPS and GMP

* unregister subscription when permission is set to denied

* cleanup after user deletion

* delete the right thing

* fix whitespace

* update install docs to include php-gmp

* suggest ext-gmp

* update javascript

* Update functions.php

Co-authored-by: Jellyfrog <Jellyfrog@users.noreply.github.com>
2021-10-06 07:29:47 -05:00
PipoCanaja
fff8b8e832
Collect OUI Database and do OUI lookups (#12842)
* first draft

refresh time

refresh time

* return codes

style

style

* presentation

* Exception details

more

fix

fix

* add tooltips

fixes for dns display

* create WebUI config option

languages

* refresh data every 7 to 11 days, keep it 15 days max

* 'Ports' and 'Port' ARP table

* Stp page support

style

style

* fix dnsname column finding when vendor is added/removed

fix dnsname column finding when vendor is added/removed

* nac vendor column

nac

* filter fields to reduce size of AJAX reply

filter fields to reduce size of AJAX reply

* fix typo on dns column detection

* default enabled
2021-05-10 14:56:48 -05:00
Tony Murray
61c89794e4
Remove debug globals (#12811)
* Remove $debug global
and $vdebug global
makes these variables more accessible and protects from collisions.

* the on boot set sends application as the first parameter, just handle that

* Relocate other debug related functions

* Log debug to stdout

* Wrong output

* remove stupid constants

* Fix lint and style issues
2021-04-29 22:42:18 -05:00
Tony Murray
0d61deb8f0
Remove perf_times table (#12517)
the data is never used... This type of data is better to store in rrd anyway
2021-02-09 12:47:05 -06:00
Jellyfrog
bc02ab3f6e
Bump to PHP 7.3 minimum (#12288) 2020-11-03 02:56:49 +01:00
Tony Murray
1e4702fa4f
Support multiple daily process locking backends with distributed polling (#11896)
* Implement locks in the file cache

* Replace custom locks

* implement restore lock
Used when re-hydrating

* remove legacy use statements

* Add class descriptions

* Fix style

* Default to database cache driver

* missed cache_locks table
prevent chicken-egg issue

* style fixes

* Remove custom file lock implementation

* missed items from file cache

* Update schema definition
hmm, other schema noise must be from manual modification as this is generated from a freshly migrated db.

* require predis, it is pure python, so no harm in adding

* and set predis as the default client
2020-10-07 07:36:35 -05:00
gerhardqux
09ae9e49bd
Do not remove users with API tokens (#12162)
* Do not remove users with API tokens

* Use eloquent

* Fix missing namespace and convert fully to Eloquent

* fix bug in purge

Co-authored-by: Tony Murray <murraytony@gmail.com>
2020-10-01 01:16:28 +02:00
Jellyfrog
29f45ca352
Apply fixes from StyleCI (#12124) 2020-09-21 15:59:34 +02:00
Jellyfrog
77c531527c
Apply fixes from StyleCI (#12117)
* Apply fixes from StyleCI

* Disable style check
2020-09-21 14:54:51 +02:00
Tony Murray
55e70be6ba Send to PHP 71/Python2 support branch 2020-05-10 23:09:03 -05:00
Jellyfrog
05507526c0
Set PHP 7.2.5 as minimum supported version (#11470) 2020-04-24 11:04:36 +02:00
Tony Murray
e4b49f59f6
Delete ports via eloquent event (#11354)
* Delete ports via eloquent event
Chunk delete during purge all operations so we don't use too much memory.

* protect against missing device

* fix whitespace

* fetch less from the database when deleting a device's ports
fix output
2020-04-16 09:19:58 -05:00
rj-taylor
c46a14af53 Prevent syslog table purge from spamming daily.log (#10851)
daily.php's syslog purge block writes output for every dbDelete() except for the final one.  The deletes are done in blocks of 1000 rows.  This can lead to verbose output while the table is purged.  This patch saves up all that excitement for a single echo at the end of the operation.  It also outputs the total amount of rows deleted.
2019-11-19 23:57:56 +01:00
rj-taylor
a28a17d963 Fixed syslog pruning when dbFetchRow() returns array (#10850)
If dbFetchRow() returns an array, dbDelete() won't do its work pruning the syslog table and we'll drop out of the block via break.  This if statement will take the value return inside the array and replace $limit with that value.  This way, dbDelete() can successfully prune the syslog table.

dbFetchCell() can return an array that won't work when fed to dbDelete().  This keeps that from occurring so the syslog table can be pruned.
2019-11-19 20:25:11 +01:00
PipoCanaja
bf181b9dc2
Added support for routing table collection in discovery (#10182)
* Clean broken VRF lite code
* Change DB table for route discovery
* Add VRF simple support
* add port_id to db and discovery
* static-fy the translation arrays
* sort and search cleaning
* Sorting refactor and validation
* formatItem shortened
* Handle ifIndex==0 meaning no next hop defined (MPLS)
* Sync all create/updates
* purge in daily
* remove old route table
* get rid of inetCidrRouteNextHop_device_id
* fix wonky column orders
* add route snmprec
* fix sorting by interface
* Move to new config
* rename to route the new table
* Properly display ipv6 compressed addresses
* Translation before merge ./lnms translation:generate
* Update manifest
2019-11-17 16:30:43 +01:00
louis-oui
0b01daa84c Do not purge alert_log table entries that have a matching active alert in alerts table (#10744)
* https://github.com/librenms/librenms/issues/10686

* Do not purge alert_logs table when there is a matching active alert in alerts table

* Do not purge alert_log table when there is a matching active alert in alerts table

* Do not purge alert_log table entries that has a matching active alert in alerts table

* Update daily.php

* Do not purge alert_log table entries that has a matching active alert in alerts table

* Do not purge alert_log table entries that has a matching active alert in alerts table

* Update functions.php

Missing }

* Do not purge alert_log table entries that has a matching active alert in alerts table

* Do not purge alert_log table entries that has a matching active alert in alerts table

* Do not purge alert_log table entries that has a matching active alert in alerts table

* Add deleting history of active alert_logs more than ? days

* Commit to replay CI

* Commit to replay CI
2019-11-05 14:49:09 +01:00
Tony Murray
36a5da00ca
Fix missed alert function renames (#10452) 2019-07-18 11:05:43 -05:00
Tony Murray
e8e6ed5142
Fix broken updates (#10380) 2019-06-26 08:49:04 -05:00
Tony Murray
f3ba8947f7
Use Config helper (#10339)
remove usage of global variable
2019-06-23 00:29:12 -05:00
Tony Murray
1a60c44eb0
Device groups rewrite (#10346)
* Device Groups rewrite
Updated web ui
Static or dynamic groups allowed
Alert rule query builder
Translation support
Permissions support

* cleanup, make relationship save, and validate it

* builder WIP

* rules builder and rules saving/loading

* Parse query builder to Laravel Fluent query

* Upgrade existing groups when editing.
Properly update only dynamic groups when polling.

* remove unused old code
Update API and other places to use Eloquent

* debug output in poller restored

* Fix up some things
creating static
improved validation
fix js error on creation
Fix static groups in polling

* hide pattern for static group

* Implement authorization
Use in the menu too

* update schema

* fix rollback

* Don't abort on invalid queries

* fixes to query builder

* add test data, looks like macros aren't handled (omitted them because groups don't use them generally)

* Add macro support for QueryBuilderFluentParser

* add test for macro that accepts value

* More space in forms
Retain rules when converted to static
no duplicate names allowed

* Better error feedback
Update related devices on save

* Add button icon

* format

* update docs

* fix tests

* Fix some QueryBuilderFluentParser issues with OR
updated/more test data

* Show device groups runtime
fix querybuilder.json format

* Store table joins in the rules to minimize polling time
Update group joins in daily.sh (and when they are saved)

* Update daily.php

* Add units to time
2019-06-19 16:01:53 -05:00
PipoCanaja
e2f835ad9d
FDB table with history capabilities (#9804)
* Migration script and data feeding

* Adding the columns in the GUI

* build schema

* update FDB test data

* Use of timestamps()

* ignore created_at and updated_at in tests, and regenerate test impacted

* daily.sh does the cleaning

* space cleaning codeclimate

* Use carbon instead of str-val

* handle when $fdb_entry->updated_at and created at are null

* handle when $fdb_entry->updated_at and created at are null (force travis rerun)

* Doc update

* Doc update
2019-02-21 13:23:01 +01:00
Tony Murray
bc93d8c037
Fix php version notification removal (#9796) 2019-02-08 08:17:02 -06:00
Neil Lathwood
6fdafd137c Deprecate PHP < 7.1.3 (#9645)
* Added validation and support for users on PHP < 7.1.3

* Branch name php56
2019-02-07 13:23:30 -06:00
Søren Rosiak
0db8d0ca70 Fix reference (#9328) 2018-10-16 08:00:50 +01:00
Neil Lathwood
3aeab9ed7d Fixed daily.sh overwriting custom alert sql (#9326) 2018-10-15 16:03:56 -05:00
Tony Murray
e9ff8c48b6 Fixed IN db queries (#9077)
Most were fine as they hardcoded the in into the query.
Change them all to use PDO properly.
Did not fix IRCBot, they are are all hardcoded.

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-08-26 13:42:21 +01:00
Tony Murray
9bc0c542a5 Allow ping checks to be ran separately from polling (#8821)
Allows ping checks at intervals not tied to the poller.  Pointless if you are not alerting on device status.
I updated the rrdstep.php script to treat ping-perf files separately and made it so it only converts if needed.

Docs here: https://docs.librenms.org/Extensions/Fast-Ping-Check/

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-07-30 22:58:38 +01:00
Neil Lathwood
d9ceaf7830 Added support for auto purging deleted ports (#8861) 2018-07-03 08:27:20 -05:00
Neil Lathwood
03076c4025
feature: Added new alert rule builder UI and rule mapping UI (#8293)
* feature: Added new alert rule builder UI

* Updated to export sql queries

* More updates

* more changes

* removed debug

* fix scrut

* Updated to include import options + various other fixes

* fix rule

* Populate name from collection rules.

* Fix default rule import
Allow new and old style rules in the collection.
Don't add new yet as I'm not sure GenSQL() is working.

* Fix GenSQL call

* Extract filter building to class so it is nicely contained in one place

* moved schema

* some fixes and tweaks

* travis fixes

* Some more features / updates

* Fix up my mistakes when adding default rules

* Use a modal for new alert (Incomplete)
Larger dialog!!
Remove page loading stuff.

Working:
Loading rules, resetting dialog, importing from collection.

Not working yet:
select width
device limited rule access? don't know what this is...

Lots of unused stuff to delete...

* reload "table" after save

* fixed editing rule

* Auto select2 width

* Reload window on save

* Restore per-device alert. Remove debug.

* Small cleanups. Rule Name first.

* Restore button to button type. Rename schema.

* Fixes: wrong command to reload window, remove extra attributes, rule is never passed

* Fixed old rule editing

* some small updates for old imports

* travis update to use trusty

* maybe travis fix

* Ability to set alert rule mappings on the rule edit screen

* pip installs one line, no quiet for deploy

* update schema def

* Fix style and some copyright headers

* fix docs missing file

* Allow new versions of snmpsim and libraries

* Parser WIP

* Fix default rules insert

* reorganize

* Legacy import first draft done

* Implement saving
Skip translation to sql for now

* Working on glues

* small rule collection fix

* Working on glues

* Working on glues

* Docs updates + small UI changes

* Parser WIP

* reorganize

* Legacy import first draft done

* Implement saving
Skip translation to sql for now

* Working on glues

* Working on glues

* Working on glues

* Add table mapping, should move to it's own class

* WIP

* Glue working!!

* Extract Schema class

* Some final touches.
revert alerts_rules.json for now.

* Finish up initial implementation
Needs more tests

* Fix a few places

* small doc updates

* Fix finding tables in grouped rules.

* remove unused code

* code format fixes

* Some quick tests for Schema
Simplified output for findRelationshipPath. Always includes start and target in the result.
This simplifies a lot of code in QueryBuilderParser.php
This also always loads the target table data now (which we want)

* Make bill_id the PRIMARY index for the bills table

* Load macros from a json file in misc instead of the database.

* Fix whitespace and wrong key for collection.

* Handle IN properly when generating SQL

* Fix glue (devices.device_id = ports.port_id) is incorrect :D
Show ALL tables we can resolve relationships for in the query builder filter.

* Remove all macros from the database
Remove insert statements, leave updates to update user's existing rules.
2018-03-14 20:25:19 +00:00
Neil Lathwood
c14efbf904
Updated email subject for failed updates to include hostname (#8258) 2018-02-16 10:08:26 +00:00
Tony Murray
d515f5043e refactor: Move install to php53 branch if running an unsupported version of PHP (#8042)
* Move install to php53 branch if running an unsupport version of PHP

* Update validate

* Move PHP version check after update check.  This will prevent us from moving no update users to the php53 branch.

Hopefully if daily.php -f update fails it returns a non-zero and we can move them to the php53 branch still.

* Wiggle things around more.

* Also, send release installs to the php53 branch.

* Set notification error for old PHP version.
2018-01-10 14:40:58 +00:00
Tony Murray
2e73b75297 fix: Use memcached to lock daily processes on Distributed Pollers (#7735)
* fix: use memcached to lock daily processes on Distributed Pollers

* All the locks!
2017-11-24 09:37:52 +00:00
Tony Murray
54b476c769
fix: devices detected as ibmtl or generic (#7618)
* fix: devices detected as ibmtl
Guess at the sysObjectID, please report if you have devices that should be ibmtl and are no longer detected.
Add output of os in discovery.

* Always load all os for discovery.  Should be cached most of the time.
2017-11-03 16:10:24 -05:00
Tony Murray
8a051aa295 fix: Fix a couple of OS cache invalidation issues (#7567)
* fix: fix a couple of OS cache invalidation issues
Fixes issues where we might be using outdated os definitions.

* use Config and fix style issue

* only pull in settings from config.php (otherwise will possibly contain populated os config)
2017-10-27 21:03:58 +01:00