Commit Graph

33 Commits

Author SHA1 Message Date
Tony Murray
229a501d4f refactor 2020-06-27 07:47:08 -05:00
Tony Murray
b573c9ef73
Implement OS specific information discovery (#11446)
* Implement OS specific information discovery
That way it doesn't have to be fetched during polling
Also improve discovery process, os is only detected once, in the core module.
EXA is the test os, a couple improvements there.

* Use local variable, then unset it so we don't pollute.

* fix style issues

* test and other fixes

* attribute update fixes

* Update exa data, need new source data

* null missing "os" values

* fix ftos odd character

* fix ftos odd character

* only null for new style or we will reset to null every discovery

* Move device observer to own class

* Handle location, relocate event logging

* update exa e7-2 data

* update ird test data, apparently unicode is now working.

* update Linux ntc, now uses correct icon

* Only load all os on the web, also, we can't load existing the the database isn't connected.

* only for devices that have a location

* revert ftos test data apparently

* revert ird
2020-05-14 11:27:59 -05:00
Tony Murray
055abcf443
Device page dropdown hero button, Performance -> Latency (#11328)
* Throw some shit together, rough outline.

* Reorganize tabs, use tab controllers

* Implement performance (into the latency tab)

* Update resources/views/device/header.blade.php

Co-Authored-By: Jellyfrog <Jellyfrog@users.noreply.github.com>

* Add more tabs

* All controllers created

* Implement routes

* Implement smokeping

* routing and auth

* fix smokeping check

* Implement device dropdown menu

* Update deviceUrl to new style

* Use Gates

* Fix style

* use more appropriate gates

* add show-config gate
remove Laravel helper

* Only show vlan tab if VLANs exist for the device :D

* Fix rancid file check will return false

* revert over-zealous file name changes

* don't need to request the location parameter, just cast to string to avoid bugs when not found

* Move latency tab (ping/performance) to the position of performance instead of ping.

Co-authored-by: Jellyfrog <Jellyfrog@users.noreply.github.com>
2020-04-29 07:25:13 -05:00
Tony Murray
df16de9d2f
Refactor Datastores to allow future improvements. OpenTSDB Tags. (#11283)
* Datastores to object oriented code, using the Laravel IoC container

Change instantiation
better DI
move OpenTSDB

Small re-orgs

remove unused stuff

Fix graphs and other scripts

Use DI for all except rrd

fix up connection error handling

Add tests, fix up a "few" things
Add Config::forget()

Style fixes

Don't reference legacy code

remove accidental code paste

Add datastores phpunit groups

some tests

* rebase fixes

* some test fixes

* shorter tests

* shorter tests

* Don't except when rrdtool can't be started.

* restore tests

* fix rrd tests

* fix iterable change upstream

* fix isValidDataset

* fix invalid data bug

* fix mysql incorrect ds

* fix issue with data that is too long

* use regular data_update()

* Use log facade

* OpenTSDB mis-ordered arguments fix

* Making a singleton with different options makes different singletons.  Just use the global config settings to disable datastores.

* only filter tags for datastores that won't it don't modify the tags permanently

* Update copyrights to include original authors.

* Stats for all datastores

* Fix mysql sends different rrd / other ds names

* fix snmp last stats not initialized
remove unused function

* remove unused function and move single use function closer to its use

* InfluxDB does not need to update null or U values.
Skip write if all fields are empty

* Fix smart value checks

* fix style issues

* Make sure port data is stored the same way as before for Graphite and OpenTSDB
Add ifIndex tag to all to be compatible

* Missed rrdtool_tune() call

* Test update WIP

* OpenTSDB now includes tags

* fix style
2020-03-16 09:17:58 -05:00
Tony Murray
7fe895bd08
Revert "Refactor datastores to classes (#9179)" (#11273)
This reverts commit e5dad7a64e.
2020-03-11 07:52:52 -05:00
Tony Murray
e5dad7a64e
Refactor datastores to classes (#9179)
* Datastores to object oriented code, using the Laravel IoC container

Change instantiation
better DI
move OpenTSDB

Small re-orgs

remove unused stuff

Fix graphs and other scripts

Use DI for all except rrd

fix up connection error handling

Add tests, fix up a "few" things
Add Config::forget()

Style fixes

Don't reference legacy code

remove accidental code paste

Add datastores phpunit groups

some tests

* rebase fixes

* some test fixes

* shorter tests

* shorter tests

* Don't except when rrdtool can't be started.

* restore tests

* fix rrd tests

* fix iterable change upstream

* fix isValidDataset

* fix invalid data bug

* fix mysql incorrect ds

* fix issue with data that is too long

* use regular data_update()

* Use log facade

* OpenTSDB mis-ordered arguments fix

* Making a singleton with different options makes different singletons.  Just use the global config settings to disable datastores.

* only filter tags for datastores that won't it don't modify the tags permanently

* Update copyrights to include original authors.

* Stats for all datastores

* Fix mysql sends different rrd / other ds names

* fix snmp last stats not initialized
remove unused function

* remove unused function and move single use function closer to its use

* InfluxDB does not need to update null or U values.
Skip write if all fields are empty

* Fix smart value checks

* fix style issues
2020-03-10 19:08:00 -05:00
Vitali Kari
532ab1127a Mpls Path Visualization (#10936)
* update nokia mibs

* database migrations

* add code

* html code

* update db schema

* add test data

* add FIXME

* move mibs/nokia/MPLS-TE-MIB to mibs/MPLS-TE-MIB
2020-01-01 23:45:06 +01:00
Jozef Rebjak
c69efb3444 Added support for Mikrotik LTE Modem (#10805)
* Added support for Mikrotik LTE Modem

* Deleted mistaken file

* Travis fix

* Another Travis fix
2019-11-13 11:31:32 +00:00
Vitali Kari
1de640c3aa MPLS Services (#10421)
* initial database migrations

* first part, discovery

* second part, polling

* add SAP polling and discovery

* style checks

* rename variables in more generic way

* html global pages for sdp and sdp binds

* add Services and SAPs

* port links, database, etc.

* html device routing pages

* add tests

* checks

* rework database id relationship

* scruntinizer inspection

* fix unit test for new mpls tables
2019-07-27 23:11:04 -05:00
Vitali Kari
00b148dbd3 Add MPLS Support (#10263)
* WIP - Add MPLS Support
This introduce the generic MPLS support
- New database tables for MPLS LSPs
- actually discovery and polling on NOKIA SR OS (Timetra) devices
- new routing->MPLS HTML pages

- ToDo MPLS LSP paths table and views
Suggestions and improvements are welcome

* add db schema

* some fixes

* add path table, discovery and polling

* add path views

* add test data

* Convert MPLS module to new style
Implement a SyncsModels trait to simplify code a lot

* abs() for negative value from snmp (bug?), test data

* fix db schema

* Fix up test data, remove uneeded data in json

* fix whitespace
2019-06-06 16:12:13 -05:00
Tony Murray
f4a33c1a34
Prevent credentials from being leaked in backtrace in some instances (#9817)
* Prevent credentials from being leak in backtrace in some instances
Particularly before the user is authenticated

* fix test
2019-03-05 00:24:14 -06:00
Tony Murray
b5714820de
Improved Exception handling (#9844)
* Update LibreNMS exception output
Use built-in laravel exception rendering.
DuskUnsafeExcpetion is triggered before Blade(view) is booted, so, try to boot it

* Cleaner blade registration

* Change 500 error page to generic LibreNMS error page.

* Handle generic exceptions from laravel better.

* Custome LdapMissingException

* Cleanup, don't need to override convertExceptionToResponse anymore

* Update unauthenticated code to current upstream style

* UpgradableException interface

* LdapMissingException extend AuthenticationException

* Code climate cleanups

* Fix code style
2019-02-21 12:08:35 -06:00
Neil Lathwood
5d74839139 Removed legacy transports and templates code (#9646)
* Removed legacy transports and templates code

* Add notification

* abstract LnmsCommand

* Only issue alerts "legacy" alerts to mail for default contacts, etc.

* Prevent no contacts error when default contact is not specified, or there are no contacts for other reasons.

* remove eval \o/

* Update notifications.rss
2019-02-15 07:58:59 -06:00
Tony Murray
587c17e215
Re-implement NAC as a class based module (#9573)
* Less sql queries for nac module

* Re-implement NAC as a class based module

* Update comments

* update module capture order

* Fix style issues
2018-12-20 19:50:12 -06:00
markoh76
7bf769e19c Ceraos more sensors (#9392)
* Update to Added support for HeliOS 10Ghz failover to 5Ghz state sensor
Converted /includes/discovery/sensors/state/helios.inc.php to yaml
Test units included
CLA has been signed

* Added support for CeraOS TX Mute Status state sensor, NTP Client Lock Status state sensor.
Updated CeraOS temperature sensor, now called from includes/definitions/discovery/ceraos.yaml - removed includes/discovery/sensors/temperature/ceraos.inc.php
Corrected issue with CeraOS temperature reverting to zero after polling (added a trailing . to the num_oid)
Amended includes/definitions/discovery/helios.yaml with changes as advised by Neil Lathwood and Tony Murray

* Added unit test data and corrected ceraos.yaml validation error

* Added support for Ceraos Wireless Sensors (php)
Cross Polar Interference
Wireless Frequency

Added Ceraos state sensors (yaml)
NTP Client Lock Status
Tx Mute Status
Radio Operational Status
Remote Radio Operational Status
IDU Temperature
Power Amplifier Temperature

* Fix up tests, etc

* remove helios changes, fix style
2018-12-11 08:41:26 -06:00
Tony Murray
3e35ee0e7d Refactored and update Location Geocoding (#9359)
- Fix location so it is a regular database relation (this allows multiple devices to be accurately linked to one location and saves api calls)
- Parse coordinates from the location more consistently
- Add settings to webui
- ~~Used [PHP Geocoder](http://geocoder-php.org/), which has lots of backends and is well tested. (also includes reverse and geoip)~~
- Google Maps, Bing, Mapquest, and OpenStreetMap supported initially.
- Default to OpenStreetMap, which doesn't require a key.  They will liberally hand out bans if you exceed 1 query per second though.
- All other Geocoding APIs require an API key. (Google requires a credit card on file, but seems to be the most accurate)
- Update all (I think) sql queries to handle the new structure
- Remove final vestiges of override_sysLocation as a device attribute
- Update existing device groups and rules in DB
- Tested all APIs with good/bad location, no/bad/good key, and no connection.
- Cannot fix advanced queries that use location

This blocks #8868

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
After you are done testing, you can remove the changes with `./scripts/github-remove`.  If there are schema changes, you can ask on discord how to revert.
2018-11-28 22:49:18 +00:00
Tony Murray
32a7c50189
Use Laravel authentication (#8702)
* Use Laravel for authentication
Support legacy auth methods
Always create DB entry for users (segregate by auth method)

Port api auth to Laravel

restrict poller errors to devices the user has access to

Run checks on every page load.  But set a 5 minute (configurable) timer.
Only run some checks if the user is an admin

Move toastr down a few pixels so it isn't as annoying.

Fix menu not loaded on laravel pages when twofactor is enabled for the system, but disabled for the user.
Add two missing menu entries in the laravel menu

Rewrite 2FA code
Simplify some and verify code before applying

Get http-auth working
Handle legacy $_SESSION differently.  Allows Auth::once(), etc to work.

* Fix tests and mysqli extension check

* remove duplicate Toastr messages

* Fix new items

* Rename 266.sql to 267.sql
2018-09-11 07:51:35 -05:00
Neil Lathwood
bc2fe6aeb1
Tidy up some transport code from previous pr (#8927) 2018-08-17 22:38:00 +01:00
TheGreatDoc
9940c884f5 Fixed snmptraps. (#8898)
* Fixed snmptraps.

* Fixed space

* Added bgp down/up and authentication failure

* Fixed typo

* Fixed some typos, arrays, astext and format_hostname

* Updated documentation

* Moved code to a function

* Some refactor

* Minor fixes

* Minor fixes 2

* More minor fixes

* Changes requested by Tony

* Minor fixes

* Moved include to snmptrap.php

* Refactor traps to use object oriented code.

Should trigger events too/instead, but we'll leave that.
Testing todo

* Add tests and fix things so they actually work
Not checking events yet.

* Fixed typo and severity level

* Update composer deps, I think the lock file wasn't right.
add json and mbstring extension deps while I'm at it.

* Fix several issues with phpunit fixtures
2018-08-11 16:37:44 -05:00
Neil Lathwood
cde15ce753
Fixed pushbullet transport (#8963)
* Fixed pushbullet transport

* Updated return value
2018-07-29 22:52:53 +01:00
Tony Murray
3c3fbd3731 feature: Show composer status in web validate. (#8181)
* Show composer status in web validate.
Don't duplicate in validate.php

* Create variable to check if a group has been completed.
No longer skips database checks.
Extract a base class.
Fix locate_binary and find_executable issues (mostly exposed by lack of db)

* Update Validator.php
2018-02-27 15:57:20 +00:00
Tony Murray
6887d771aa device: Added CeraOS wireless sensors (#8167)
* CeraOS wireless sensors

* Add CeraOS test data, fix MSE

* fix whitespace
2018-02-18 21:31:18 +00:00
Tony Murray
5141fc4872 refactor: Share code between all mysql based authorizers (#8174)
* Share code between all mysql based authorizers
I plan to update the mysql password encryption and this will allow the code to be changed in a single location.
It also reduces a lot of duplication.

* Fix tests, I suspect reauthenticate will work for these...
Do not allow password updates for several authorizers
2018-02-06 21:20:34 +00:00
Tony Murray
11147d3bbf
Major Processors rewrite (#8066)
* Extract DiscoveryItem and move some things to better places.
Extract model class
Fix up model construction.  I have problem with construction...
Makeshift model working.  Switch constructor to factory.  discover() and create()
Support legacy discovery.
Remove uneeded custom pollers
Remove netonix custom detection as we try ucd on all os now.
Add a few yaml procs.  Fix a couple things.
More processor discovery conversions
Move Calix e7 to standard hrProcessorLoad, but it doesn't fully implement the HR-MIB, move things around to make it work.
Add a few yaml procs.  Fix a couple things. Correct some stupid mib stuff.
Move more, drop php 5.3
Add netscaler which uses string indexes.  Port fiberhome to yaml and use skip_values
More conversions.  BroadcomProcessorUsage Trait
Serveriron and Ironware share some mibs.  Create a common abstract os for them.
Add yaml support for mib specification in each data entry
Make legacy discover_processor() set 0 for hrDeviceIndex

Untangle Dell switch OS processors

Use use shared OS for groups if they don't have a specific group.
fix silly mib mistake

Make index optional

Move HR and UCD to Traits and out of Processor.

* forgot to update the fortiswitch index

* Make sgos and avaya-ers match the old index.

* fix comware test data

* fix merge errors

* fix dsm and remove pointless empty modules

* file not found exception is in the wrong place.

* Updated processor development docs
2018-02-05 07:39:13 -06:00
mcq8
5ff03d5942 refactor: Refactor alert transports to classes (#7844)
* Refactor alert transports to classes

* Fix linting

* Make Slack alert transport php 5.3 compatible

* Rename call method to deliverAlert for alert transport
2017-12-10 20:20:28 +00:00
Paul Heinrichs
0dd3ec25d3 device: Update Cambium devices to the new wireless sensors (#7703)
* Inital cambium refactor

* Refactored tests

* Remove CMM from pmp group

* Add sensors / Give CMM individual OS

* Fix linting

* More Sensors / states and Revised Headers

* Added old graphs / more sensors

* Add modulation rate to sensor

* Fix Hardware detection bug

* Further hardware detection refinments

* Modify pmp hardware detection

* New Wireless Metric - SSR

* SSR Sensor Graph

* Migrate SSR to metric

* Added last metric file

* Add master check to isAp method

* Fix variables in epmp

* CMM Discovery and state addition

* Add another test for cmm
2017-12-06 21:36:01 +00:00
Adam Bishop
1c6b7a967f Single Sign-On Authentication Mechanism (#7601)
* Allow the URL a user is sent to after logging out to be customised
This is required for any authentication system that has a magic URL for logging out (e.g. /Shibboleth.sso/Logout).

* Allow auth plugins to return a username

This is a bit cleaner than the current auth flow, which special cases e.g. http authentication

* Add some tests, defaults and documentation

* Add single sign-on authentication mechanism

* Make HTTPAuth use the authExternal/getExternalUsername methods

* Add to acknowledgements

* Add reset method to Auth
2017-11-28 20:40:17 -06:00
Tony Murray
2e73b75297 fix: Use memcached to lock daily processes on Distributed Pollers (#7735)
* fix: use memcached to lock daily processes on Distributed Pollers

* All the locks!
2017-11-24 09:37:52 +00:00
mcq8
c9728a1f71 refactor: Refactored authorizers to classes (#7497)
* Refactored authorizers to classes

* Merge changes for #7335

* ! fix php 5.3 incompatibility

* Update ADAuthorizationAuthorizer.php

* Fix get_user -> getUser

* Rename AuthorizerFactory to Auth, fix interface missing functions

* Add phpdocs to all interface methods and normalize the names a bit.

* Re-work auth_test.php AD bind tests to work properly with the new class.
Reflection is not the nicest tool, but I think it is appropriate here.
Handle exceptions more nicely in auth_test.php

* Restore AD getUseList fix

Not sure how it got removed

* fix auth_test.php style
2017-11-18 10:33:03 +00:00
Tony Murray
51ba934e11 feature: Added web validation (#7474)
* refactor: convert validations to modules to prep for gui integration

* accidentally dropped, schema update

* fix accidental output to webui

* mention discovery-wrapper.py and re-arrange.

* refine some fix statements

* rename the Config validation group to Configuration.

* fix some scrutinizer issues
remove as many local functions from validator.php as possible
move extensions from pre-check
remove duplicate timezone check
looks like there is some db schema differences between mariadb 10.1 and 10.2, investigating

* mariadb version diff take2

* Check schema version first for database.
Remove stop to go back to command line for install docs.
Add helpful link when there is no devices added to /addhost

* fix incorrect validation for empty string defaults

* Fix style

* Add additional file permissions checks

* catch exception and fail for invalid timezone
Change visuals a bit.

* add php version warning

* fix space
2017-10-26 07:56:09 +01:00
Tony Murray
527a989b4c newdevice: Added wireless sensors for SAF Tehnika (#6975)
Add new sensor type MSE (Mean Squared Error) which is similar to SNR
2017-07-07 16:05:55 +01:00
Tony Murray
5ca07e11aa newdevice: Added connected AP count for Cisco WLCs (#6572)
* newdevice: Add connected AP count for Cisco WLCs
Adds ap-count wireless sensor type
fixes: #4295

* better detection
Update the overview graph to the new one
2017-05-04 08:03:10 +01:00
Tony Murray
4c0412b14d feature: Wireless Sensors Overhaul (#6471)
* feature: Wireless Sensors
Includes client counts for ios and unifi
Graphing could use some improvement.
Alerting and threshold ui not implemented

WIP: starting OO based wireless sensors.

Class based functionality working
remove old functional files
add schema file

discovery needs to be enabled, not polling

fix up schema

fix Unifi discovery not returning an array

Add some debug when discovering a sensor.
Fix style.

Add missing semicolin

Add a null object (Generic) for OS.
Fill out some phpdocs

Re-organized code
Each sensor type now has it's own discovery and polling interface
Custom polling tested with Unifi CCQ

Left to do:
Implement UI (Graphs and Custom thresholds)
Alerting
Testing

Fix event message text

Remove runDiscovery and runPolling from OS, they are unused and don't belong there.

Cleanups/docs

Missed this file.

Remove the requirement to fetch the current value to check validity.
Do that automatically if current is not specified
A few cleanups here and there

First pass at graphing.
device_ and wireless_ graphs added.

Add RouterOS support

Singleton OS instance isn't required right now.
Remove that to allow some memory to be freed.

Add wireless to the device list metrics.
Make all metrics clickable

Tweak graphs a bit

Implement limit configuration page.
Use sensors page as common code instead of duplicating.
Clean up some javascript interactions:  Allow enter on values to save. Cancel if update is not needed. Enable the clear custom button after setting a custom value.
Add some wireless alert rules to the library.

Add documentation.

Add unifi client counts by ssid in addition to radio.
Optimize Sensor polling a bit.

Add HP MSM clients support (for full controller)
Fix function accessibility

Formalize the discovery and poller interfaces.

Add Xirrus clients and noise floor
move module interfaces to a more appropriate place.
push caching code up to os, unsure about this do to the limitations

No point in selectively enabling wireless discovery.  We only discover if the device supports something.

Add RSSI, Power, and Rate.
Add these sensors for Ubnt Airos.
Clean up some copyrights.

Reduce the amount of files need to add new types.
Leave graph files for consistency and to allow customization.

Remove the old wifi clients graph completely.
ciscowlc should have improved counts (total and per-ssid)

Schema didn't get added.

Impelement the rest of the AirOS sensors
Reformat and re-organize the Airos.php class.

Add several UBNT AirFiber sensors

A few fixes add links to the section headers

Add HP MSM mibs.

* Schema file got dropped in rebase.

* Add wireless menu to view sensors across all devices.
Icons in the menu need help :/

* Add HeliOS, Mimosa, and Siklu support
Sensors added SNR + Noise

* Add power and utilization to Unifi

* Update polling to prefetch all sensor data in a few snmp requests as possible

* Add Extendair: tx+rx power, aggregate rate, frequency

* Add a check for duplicate sensors in discovery.  Just print an error for now.

* Add Bit Error Ratio (named error-ratio to allow for bit error rate to be added if needed)
Fix an incorrect link in the wireless sensors table

* Add error rate and change all bps and Hz to use si units

* Fixes to limits and frequency display

* Fix overview graph frequency display
A few decimal place tweaks

* Don't allow switching sensor and wireless-sensor graphs, it doesn't work.
Change individual distance graphs to use si units

* Go through the OS and make sure I got all the sensors I can (probably missed some still)
Because pollWirelessChannelAsFrequency() is generic and a little complex, so pull it up to OS.
Message to help developers adding supports that don't return an array from discover functions.

* Fix some issues

* Remove noise and signal for now at least
A couple more fixes
Add a notification

* Oopsie

* Bonus AirFiber sensors
2017-05-01 23:49:11 -05:00